Method of and system for inserting/overlaying markers, data packets and objects relative to viewable content and enabling live social networking, n-dimensional virtual environments and/or other value derivable from the content

ABSTRACT

A method by which a User can associate selectable Markers, Data Packets and/or Objects with Content. The Content may generally be distributed electronically, and the Markers allow for insertion and/or overlay of Objects when the Content is selected for viewing by a Viewer. Objects and Data Packets are generally provided by a User, Promoter, Host, Service, or other entity to convey information to a Viewer. A Service provides tools and capabilities to both the User and the Promoter to facilitate their respective actions according to embodiments of the invention, including enabling the creation of live social networks (such as those linked to a specific Service provider, a specific User group, activities by a specific Promoter, and/or to specific Data Packets) and the creation of n-dimensional Virtual Environments.

RELATED APPLICATIONS

The present application is a Continuation of each of, and claims thebenefit of priority to each of, co-pending U.S. patent application Ser.No. 12/004,622 filed 20 Dec. 2007 and co-pending U.S. patent applicationSer. No. 12/004,392 filed 19 Dec. 2007, each of which is in turn aContinuation-in-Part of and claims the benefit of priority to:

-   -   a. U.S. patent application Ser. No. 09/686,206 filed 10 Oct.        2000, now abandoned;    -   b. U.S. patent application Ser. No. 09/749,091 filed 26 Dec.        2000, now abandoned;    -   c. U.S. patent application Ser. No. 11/982,707 filed on 2 Nov.        2007, now U.S. Pat. No. 8,117,281 issued 14 Feb. 2012;    -   d. U.S. patent application Ser. No. 11/477,162 filed 28 Jun.        2006, now abandoned; and claims the benefit of priority to:    -   e. U.S. Provisional Patent Application Ser. No. 60/877,891 filed        28 Dec. 2006, now expired; and    -   f. U.S. Provisional Patent Application Ser. No. 60/918,984 filed        19 Mar. 2007, now expired;        the disclosures of each of which are incorporated herein in        their entirety by this reference. The present application also        incorporates herein by this reference the entire disclosures of        each of U.S. Provisional Patent Application Ser. No. 60/856,404        filed 2 Nov. 2006, U.S. patent application Ser. No. 09/745,257        filed 20 Dec. 2000 and issued as U.S. Pat. No. 7,168,051 on 23        Jan. 2007, and U.S. patent application Ser. No. 09/384,606 filed        on 27 Aug. 1999 and granted as U.S. Pat. No. 6,553,566 on 22        Apr. 2003.

FIELD OF THE INVENTION

This invention relates generally to the field of electronicallytransmittable information. More particularly, it concerns supplementingcontent which is viewable via electronic devices so that additionalinformational content is available to content viewers to enablecommerce, brand building, communications, and social networking, amongother things.

BACKGROUND OF THE INVENTION

Viewable content available today, such as that downloadable from theinternet and/or created and distributed by a wide variety of sources,generally contains and conveys to viewers a limited amount ofinformation. As much information as the Content has at the time ofcreation, it generally consists of that same amount of informationthroughout its existence. Content may be edited from time to time tocreate new content, but generally the quantity and quality of thecontent remains relatively constant (e.g., static, unchanging).

Content creators can generate a revenue stream by the sale, rental,licensing, or other provision of content to consumers of content, but acontinued revenue stream largely depends upon continued creation anddistribution of new content. Content creators may also generate somerevenue through ‘product placement’, in which merchandisers pay contentcreators to place their merchandise prominently in content. For example,the Ford Motor Company and/or Coca-Cola, Inc. may pay a movie producerto have a main character in a movie drink Coca-Cola branded beveragesand drive a Ford Mustang convertible. However, once the movie is filmed,edited, and distributed, no more revenue can be obtained from productplacements within that movie content. The content is static in quality,and has a limited revenue generation curve.

Viewers also receive only static content, which becomes uninterestingonce viewed one or more times, and may in fact be uninteresting from themoment of creation. Viewers must search through a multitude of contentsources to find content that is interesting and relevant to them.Separately, they shop, interact, gather news, and carry on many otheractivities, each activity relatively independent from each other. Inthis age of proliferating information, viewers consume tremendousamounts of time pursuing information that is personally relevant andinteresting.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1 a-1 b are block diagrams depicting a plurality of operationsincluding content upload and registration according to an embodiment ofthe invention.

FIG. 2 is a block diagram depicting a plurality of operations includinga promoter making objects available for inserting and/or overlayingaccording to an embodiment of the invention.

FIG. 3 is a block diagram depicting a plurality of operations includinginserting and/or overlaying objects according to an embodiment of theinvention.

FIG. 4 is a block diagram depicting a plurality of operations includinga viewer selecting an object according to embodiments of the invention.

FIG. 5 is a block diagram depicting a plurality of operations includinginserting and/or overlaying attribute relevant objects according toembodiments of the invention.

FIG. 6 is a block diagram depicting a plurality of operations includinginserting and/or overlaying objects relative to distributed and/oredited content according to embodiments of the invention.

FIG. 7 is a block diagram depicting a plurality of operations includingreviewing content for objectionable material according to an embodimentof the invention.

FIG. 8 is a block diagram depicting a plurality of operations includingflagging content according to embodiments of the invention.

FIG. 9 is a block diagram depicting a plurality of operations includingapplying digital rights management to content according to an embodimentof the invention.

FIG. 10 is a block diagram depicting a plurality of operations includinga promoter altering capability for inserting and/or overlaying objectsaccording to an embodiment of the invention.

FIG. 11 is a block diagram depicting a plurality of operations includinga promoter replacing an object with another object in a stored selectionaccording to an embodiment of the invention.

FIG. 12 is a block diagram depicting a plurality of operations includingassociating code relevant to a virtual environment with contentaccording to an embodiment of the invention.

FIG. 13 is a block diagram depicting a plurality of operations relevantto a virtual environment according to embodiments of the invention.

FIG. 14 a is a screen-grab depiction of viewable content according to anembodiment of the invention.

FIGS. 14 b-e are screen-grab depictions of viewable content includinginserted/overlaid objects according to an embodiments of the invention.

FIGS. 14 f-g are screen-grab depictions of viewable content includingselectable objects in n-dimensional virtual environments and/or worldsaccording to an embodiments of the invention.

FIG. 15 is an iconic system diagram depicting aspects of a system forcreating, distributing, hosting, and/or viewing content having enhancedinformational and/or other value according to an embodiment of theinvention.

FIG. 16 is a block diagram depicting storing historical informationderived from interactions relative to social networks and/orn-dimensional virtual environments, and uses of such information,according to an embodiment of the invention.

FIGS. 17 a-s are screen grab depictions of a music content-basedembodiment of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The invention in accordance with a preferred embodiment involves asystem and method by which a User can place selectable Markers intoContent. The Content. may generally be distributed electronically, andthe Markers allow for insertion and/or overlay of Objects when theContent is selected for viewing by a Viewer. Objects are generallyprovided by a Promoter to convey information to a Viewer. A Serviceprovides tools and capabilities to both the User and the Promoter tofacilitate their respective actions, as described in detail below.Viewers can be linked concurrently with other Viewers in live socialnetworks that are created and based substantially upon the same, similarand/or specific attributes of Markers, Objects and/or Data Packetsincluded in content. Live social networks may be linked to a specificService provider, a specific User group, activities undertaken by aspecific Promoter, and/or to specific Data Packets.

For purposes of clarity, general definitions are provided for some ofthe terms used throughout this description. These definitions are notintended to limit the scope of each term, but only to provide an initialunderstanding so that the reader can readily recognize and understandthe terms when used herein. Those having ordinary skill in the art willrecognize from the overall specification that each term has a moreexpansive scope according to alternate embodiments than is contained inthese brief summary definitions.

“Content” includes any viewable content, including images, whetherstatic (e.g., still photographs, artwork) or non-static (e.g., video,animation), whether original (e.g., user-created) or not (e.g., obtainedby User from another source), interactive Content, such as n-dimensionalVirtual Environments and n-dimensional Virtual Worlds, whetherelectronically transmittable to a Viewer (e.g., via internet) ortransferred to a Viewer on an electronic storage medium (e.g., a digitalvideo disc (DVD)), audio or text, that may be viewed on aninternet-connected (or connectable) electronic device. Examples ofContent include video, video games, still images, any motion content,video conference calls, live video feeds, n-dimensional virtualenvironments and/or worlds, etc., blogs, podcasts, and vlogs. Nor shouldContent be considered limited to those types listed here, but canencompass all presently available and/or foreseeable forms of electronicmedia and/or content that is or can be visually displayed. Contentincluding an associated marker and/or data packet may be referred to as“Marked Content”, and can exist as a separate content layer overlying,corresponding to, and dynamically responsive to a layer of viewablecontent.

A “User” is an entity (e.g., individual, company, organization,governmental body) that provides Content that may be accessed and viewedby a Viewer. A User is generally human, but may include electronicsystems or devices designed to perform all or some subset of theoperations that a User performs as described herein. For example, a Usercan be a camera or video monitor coupled with pattern and/or imagerecognition software for tracking, monitoring, cataloguing, archiving,and/or reporting Content. An ‘n-User’ is any User who edits alreadymarked Content, whether or not the n-User has registered with theService.

A “Viewer” accesses Content provided by a User. A Viewer is generallyhuman, but may include electronic systems or devices designed to performall or some subset of the operations that a Viewer performs as describedherein. For example, a Viewer can be a camera or video monitor coupledwith pattern and/or image recognition software for tracking, monitoring,cataloguing, archiving, and/or reporting Content. A Viewer may notactually View Content visually, but may access the Content nonethelessand be considered a Viewer. Further, identification as a Viewer is notlimited by whether Objects inserted and/or overlaid at Markers withinthe Content are viewable, or rather may convey primarily, partially, orentirely information of some other sensory nature (e.g., audio). In someembodiments, a Viewer can also be a reader or a listener.

A “Promoter” is any entity (e.g., individual, company, organization,government) with the intent to provide information and/or content (e.g.,brands, products, etc.) to Viewers. A User, Host and/or a Service canalso be a Promoter in some embodiments of the invention.

A “Marker” is generally software (e.g., device executable code)configured to be associated with (e.g., inserted into or overlaid upon)Content. A Marker may occupy a relatively fixed position relative to theviewable Content, or it may be associated with the Content so that theMarker's position is variable relative to the viewable Content. Markerscan be generic shapes and/or symbols representing an insertion/overlaypoint for an Object to be inserted and/or overlaid relative to a Contentimage. A Marker may be either visible or invisible to a Viewer whenviewing Content. A unique identification code can be associated with orencoded into each Marker, and can be used to associate the Marker withparticular Content, Promoters and/or Objects. A Marker can provide a‘placeholder’ and/or ‘insertion/overlay’ point in Content for theplacement of one or more Objects.

An “Object” is any piece of information that a Promoter wishes viewersto access. Objects can include brand names, brand objects, messages(audio, visual, text, or links thereto), although the embodiments arenot in any way limited to this partial list. Objects can include audioor other information that may be either partially or entirelynon-viewable, yet conveys information and/or data to a viewer. Objectscan also be associations, links (e.g., hyperlinks), or virtual conduitsthat link the Viewer to informational content provided by a Promoter. AnObject can also be a hotspot including data linking a Viewer with otherViewers and/or social networks. An “Object” can also provide access toan n-dimensional virtual environment or n-dimensional virtual world thatViewers can view and/or enter, and that users can be transported ortravel through to other n-dimensional virtual environments orn-dimensional virtual worlds. Objects can be either persistent (e.g.,permanent) or temporary virtual images, and can be changed on a computeror on a server. Objects can be changed remotely (e.g., via the internet)or placed at (e.g., associated with) a Marker in Content at any time bythe source of the Content (e.g., a User), by a third party, or by acomputer program or computer/internet-based system for placement ofObjects. Objects can be specific to a geographical area where a Vieweris located, or specifically targeted to the demographics of a Viewer. AnObject may be either visible or invisible to a viewer, and may be avirtual environment, or may link to a virtual environment. Objects mayalso be self-executing (e.g., software code) when Content is selectedfor viewing, viewed, stored, or otherwise operated upon by a Viewer.Therefore, a self-executing Object may not require a Viewer to selectthe Object as a prerequisite to obtaining information from the Object.

A “Service” provides tools and/or capabilities to a user enabling theuser to associate Markers with Content. A Service also provides toolsand/or capabilities to a Promoter enabling the Promoter to insert and/oroverlay objects at Markers in Content. A Service can detect and trackaccess to marked Content and Objects to assign rights to compensation(e.g., payment) to Users, Promoters, Hosts, the Service, or others. Whena Promoter, Host or other third party is able to perform activitiesaccording to embodiments of the invention, perhaps using tools,capabilities, or authority provided by the Service, the Promoter, Hostor third party so acting may also be considered ‘the Service’ withregard to those actions. In general, the tools referred to herein areprimarily or entirely software-based tools configured for execution onor by a device that can be connected, either directly or indirectly, tothe Internet or another network (e.g., an internet-linkable or linkeddevice).

A “Host” is an internet-accessible repository for Content (e.g.,website) at or through which Viewers can access Content provided byUsers. Examples of Hosts could include Google Images, YouTube, CNN.com,or virtually any other at which viewable Content may be made availableto a Viewer by a User. The number of potential Hosts is virtuallyinnumerable, and increasing nearly every day.

A “Social Network” is a social structure generally comprising aplurality of interconnected nodes, each of which are typically, althoughnot exclusively, individuals or organizations. Social networking canrefer to a category of internet applications that help connect friends,business partners, or other individuals together using a variety oftools. A “live social network” is a Social Network in which membersengage with each other concurrently, so interactions in a live socialnetwork are substantially as responsive and real-time as interactionsbetween people meeting face-to-face.

“Metadata” is information about data and/or other information. Metadatais typically structured, encoded data that describe characteristics ofinformation-bearing entities to aid in identifying, locating,discovering, assessing, and managing the described entities. Metadatadescribed herein can also include “keywords”.

A “Data Packet” is typically a discrete and complete unit of executabledata configured to be associated with and/or relative to Content. Inembodiments, a Data Packet may include compressed or uncompressedinstructional data and/or an audio/video packet and can be associatedwith content by an n-User, a Promoter, a Host, a Service, or by anotherentity. A Data Packet can be recognized by software and segregated toreveal compressed or uncompressed audio/video/instructional/executabledata, or an n-dimensional virtual environment and/or world that Viewerscan view and/or virtually participate within. A Data Packet can also beplaced within, and be used as a portal from one n-dimensional virtualenvironment and/or world to another or to other Content. A Data Packetmay subsequently be executed, or stored for later execution, filing orretrieval, and/or the Data Packets could be viewed on the screen of atarget viewing and/or listening device (e.g. a desktop, webtop,television, computer, mobile telephone, electronic book, video gameunit, Personal Digital Assistant, radio, internet radio, a networkeddevice). Data Packets can be self-executing, so that the data in apacket executes immediately when the Content is viewed. Alternatively, adata packet may execute at the completion of a predetermined delay(e.g., predetermine and set by the User) after a Viewer begins viewingContent. Additionally, a Data Packet may execute and run in a separatewindow on a User's device display, with or without instruction from theviewer. A Data Packet could enable messages to be placed within Contentand to be retrieved by an n-user.

A “virtual environment” is a computer simulation of a real or imaginaryenvironment. A virtual environment may allow people to enter andinteract within n-dimensional computer graphics worlds. A “virtualworld” is a computer-based simulated environment intended for its usersto inhabit and interact via avatars. This habitation usually isrepresented in the form of two or three-dimensional graphicalrepresentations of humanoids (or other graphical or text-based avatars).Some, but not all, virtual worlds allow for multiple users. An“augmented reality” is the combination of real world and computergenerated data. These 3 terms are. collectively referred to herein as“Virtual Environment” or “virtual environment”.

Numerous references to ‘the Internet’ occur throughout this description.While such references most typically refer to the World Wide Web, as hasbecome broadly understood and used by the general public, theembodiments are not so limited. Embodiments of the invention couldfunction and provide benefits equally or substantially as well whenimplemented on or with various limited access and/or entirelyproprietary networks, or other systems of electronically interconnecteddevices. Examples of networks according to alternative embodimentsand/or applications could include Arpanet, proprietary intranet systems,heavily censored national networks (e.g., as in China), and others.Therefore, reference to ‘the internet’ herein are to be construed toinclude any network with which or within which a service can beinterconnected, or into or from which Content is published, downloaded,uploaded, or otherwise conveyed between a User and a Viewer.

Although much of the Content referred to throughout is described asContent transmittable via the internet, Content-based technologies haveand continue to rapidly converge. For example, Content conveyed viatelevision signals can be received by and displayed on a display meansof various devices (e.g., computers), or can be conveyed over telephonenetworks via fiber optic systems, just to name a few examples.Therefore, references to Content and systems throughout this descriptionare not limited to Content conveyed primarily or wholly through internetor computer readable media, but may also include Content conveyed atleast in part via systems typically associated with other purposes(e.g., broadcast, communications, entertainment, data transfer,navigation). Virtually any system or network by which Content can beconveyed between a User and a Viewer, and to, through, or with which aService has either a persistent or occasional connection, is alsocontemplated within embodiments of the invention. Additionally, a‘display means’ herein includes any device or portion thereof configuredto render content visually perceivable by a Viewer (e.g., liquid crystaldisplay, cathode ray tube, plasma screen, light emitting diode display,etc.)

Use of the term ‘may’ (e.g., ‘may’, ‘may be’, ‘may have’) within thisdescription generally indicates that the described operation, structure,or other innovative element described is present in one or moreembodiments of the invention, but is not necessarily present in allembodiments. Therefore, the term ‘may’ is not indefinite herein, butindicates a condition relevant to at least a subset of the possibleembodiments according to the applicants' contemplation of the invention.

Additional terms appear throughout this description, whose meaning willbe apparent to those having ordinary skill in the art, based on theirusage in context and/or by reference to the figures.

1) Content Upload and Registration

FIGS. 1 a-1 b depict a number of operations according to an embodimentof the invention. Although these operations are depicted in series, someindividual operations may be omitted, transposed, or may occur inparallel in alternative embodiments.

As shown at 102 and 104, a User creates viewable Content, and uploadsthe Content to a Host. If the Host is a website, the User may upload theContent to the Host's servers. Alternatively, the User may simply createa hyperlink at the Host website that viewers can select to access theContent located elsewhere. The User may choose to maintain the Contenton an internet-linked server, computer, or website the User controls, inwhich case the User may also be considered a Host.

If the User broadcasts from his own website, server, or computer, andwants to use the Service directly, the User can proceed directly to theoperation shown at 110 in FIG. 1A, and transfer the Content to theService servers. However, if the User chooses to use an external host,the Host can ask the User at 106 if he would like to add selectableObjects to the Content by use of the Service. The User may wish toinsert objects for his own purposes, for example personal messages,pictures, or commentary on the Content. Alternatively, a Host may alsoask if the User would like to generate revenue from Promoters that willpay to have their Objects (e.g., brand names, brand objects, oreCommerce links) associated with the Content through the Service.Generally, the Host will provide a method by which the User can registertheir affirmative or negative answer to the Host's query.

If the User wants to use the Service, he selects the providedaffirmative option, at 108, and the uploaded Content is transferred tointernet (web) servers under the control of the Service, as shown at110. At 112, the Service prompts the User to register his personalidentification data with the Service, and the User submits the requesteddata for registration. Such information may be retained by the Serviceon user ID servers. User personal identification data that may becollected includes standard locator information (e.g., name, address,phone number, email address), various information related to the User'sinterests, User demographic information (e.g., age, sex, place of work,income level), and frequently visited web sites, although the scope isnot so limited. The Service can use such demographic and otherinformation to specifically target and provide Promoter information,products and services to Promoters' targeted groups. For example, if ashoe company wants to reach or have their products be marketed to aspecific market, such as hikers or skateboarders, the Service can usedemographic information to direct those products and services to thatmarket. In addition to information provided by the User in response tothe Service queries, the Service may additionally have tools to identifywhat other websites the User has visited, or what videos or audios theUser has downloaded, to enable the Service to build a more detailedprofile of the User's interests.

The Service queries the User, at 114, to identify the source or methodby which he was referred to the Service. The User will be given theoption to name who (e.g., an individual) or what (e.g., company,website, service) introduced him to the Service. This information willenable the Service to share a portion of any revenue that is generatedfrom such referral, for at least a specified time period. Alternatively,if the User's content was transferred to the Service from a Host, as at110,, the Service may recognize the Host as the referral source withoutquerying the User.

2) Association of Markers and/or Data Packets with Content

The User will be instructed to review his Content and place Markers intowhatever portion of the viewable Content (e.g., image) seems appropriateto the User. For example, if the Content is a video, the User may beinstructed to play the video in slow motion, or at variable speed, andto place Markers at locations in the video. In general, the User mayhave a graphical user interface (GUI) that resides on his computer‘desktop’, on a ‘webtop’, and/or on the website of the Host to which hehas uploaded his Content. The Service will provide simple ‘drag anddrop’ and/or ‘point and click’ and/or ‘outlining’ tools to placeMarkers. For example, the User may use a ‘mouse’ peripheral device tosteer a position indicator on the GUI display to a particular positionwithin the Content. By clicking a mouse button, the tools provided bythe Service will associate a Marker with the image at the locationselected and indicated by the User. The assigned position of a Markerspatially with regard to the apparent dimensions of an image (e.g., 2-ndimensional), and, temporally relative to a beginning or an end of thecontent (in the case of non-static content), is retained by the Serviceservers for future processing. In an embodiment, when marking Contentcreated by the User, or Content for which the User possesses validrights and/or permission to alter, the User can directly modify theContent by adding Markers. However, if the User is not authorized toalter the Content, or does not wish to alter the Content, association ofMarkers with Content creates an additional layer of Content which isseparate from, but is identified to, correlated (perceivably registered)to, and dynamically responsive to the original Content. In eithersituation, the Content with which the Markers are associated with isreferred to herein as “Marked Content”.

The User continues to associate Markers at selected positions throughoutthe Content subject to the User's judgment, or subject to guidelinesestablished by the Service. It will be clear to those of ordinary skillin the art that while a computer mouse peripheral device is one meansfor indicating and selecting locations for placing Markers, theembodiments are not so limited. For example, the User may indicate alocation for associating a Marker by touching a handheld stylus, theUser's finger, or some other pointing directly on a touch-sensitivedisplay screen. Likewise, other systems and devices such as gazetracking systems, sip and puff control devices, joystick controldevices, voice-recognition control systems, and numerous others canlikewise be used to indicate a location and associate a Marker with theContent. In addition, a pattern recognition and/or image recognitionmeans (e.g., embodied in software, firmware, or hardware, or acombination thereof) could be used to define, locate and automaticallyassociate Markers within the Content. For example, an image recognitionprogram could be used to place a Marker on every shirt in a video forthe subsequent placement of Objects on those shirts.

During Marker placement, Markers can be visible to the User, to aid theUser in properly positioning the markers relative to the Content. Forexample, Markers can be assigned as 2-dimensional regions defined and/ordelineated by the User using tools similar to a ‘telestrator’ (commonlyused in sportscasts and broadcast weather reports). By this method,Users define a boundary or outline, as shown at 1410 in FIG. 14 b,corresponding with a particular viewer-perceivable pattern or object(e.g., a wall, clothing, a vehicle) in an image, and can choose toassociate the Marker with that pattern. Pattern recognition software canthen track the object pattern as it moves through, past and relative toother patterns in the Content, and maintain the Marker associated withthe pattern. If for example, the Content is a video, and the patternwith which a marker is associated is a car, when the car moves out ofthe visible range (e.g., viewer-perceivable range, field of view) in theContent image (e.g., off the display to one side, or shrinking into thedistance), the Marker may likewise progressively move out of or shrinkbeyond the visible range. If the pattern then returns within the visiblerange of the Content, the Marker likewise returns. Alternatively,Markers can be persistent, and remain within the visible range of theContent even when an associated object or pattern in the Content nolonger appears within the visible range.

Markers may also have predefined attributes, such as shape, size,opacity/transparency, or color, which may be either fixed (notchangeable by the User) or which may be User adjustable. Such attributescan also be adjusted by operation of a pattern recognition means tomaintain a user-perceivable correlation between viewer-perceivableattributes of the Marker and those of an associated viewer-perceivableelement of the Content (e.g., a shirt, a bottle, a car, etc.). Markersmay be included in the tools provided by the Service, or they may beavailable as a part of the User's or Host's GUI. The Service can alsoprovide Markers with predetermined attributes corresponding to commonitems, such as beverage bottles, signs, or others. Likewise, Users,Promoters, and others can create new Marker configurations and submitthem to a Service. When those Marker configurations are then used byothers, the creator of the Marker configuration can receive a smallroyalty payment. The possibilities are relatively limitless, and newMarker designs could also be made available to Users at any time asupgrades and/or revisions to the provided tools. Marker configurationscould also be a revenue source of a Service, in that Users may pay touse interesting Markers for their Content similar to how mobile phoneusers currently pay to download interesting ring tones for their phones.Users, promoters, or other third parties could also create interestingmarkers and provide them to a Service in exchange for recognition ofsome kind of consideration. The Service can then make these Markersavailable to others to use and similarly collect a use fee.

The Service will prompt the User and the User registers the Markedcontent, at 118, and at 120, a unique identification is associated withthe Content. This unique identification, among other things, allows theService to recognize the Content when a Viewer selects the Content forviewing. The unique ID also helps the Service maintain association ofMarkers, and insertion/overlay of Objects, both of which may be storedat Service servers (e.g., on a hard drive, data storage tape, opticaldisk, or another data storage means), with the proper Content, even whenthe Content itself resides elsewhere. 100531 The User is then asked, at122, to provide and/or define Content-descriptive values, or keywords,which describe the nature, origin, general theme, or othercharacteristics of the Content. These keywords and/or values will assistsearch engines (e.g. Google) to place text-based advertisements with theContent, (which, for example could be placed at the end of a video clipor displayed beneath it), or vice versa.

At 124, the Service prompts the User to suggest which brand names, brandproducts, or other Objects would be best to insert and/or overlay ateach Marker in the marked Content. The Service can also provide a listof brand names, products or other Objects from which the User can selectappropriate choices, including those submitted to the Service by variousPromoters who wish to reach target audiences through User-markedContent. For example, if the User suggests using the brand name PEPSI toinsert and/or overlay at a Marker associated with the image of a bottleon a table in the User's content, the brand PEPSI can be made to appearon the bottle whenever a Viewer selects the Content for viewing, and thebottle appears within the visible range in the Content.

As previously described, pattern recognition software can recognize thesize, shape, color, and other visual characteristics of a selectedobject or pattern (e.g., the bottle) in the Content. The software canalso make adjustments to keep the position of the Marker relative to theposition of the bottle relatively constant in the Content image, as theposition of the bottle in the image changes based on the changingposition of the image capture device (e.g., camera that originallycaptured the Content image(s)) relative to the bottle. Further, thepattern recognition software can also make adjustments to the lightingand apparent perspective. For example, if the bottle is placed upon atable, and the camera moved in a full 360-degree circle around thebottle while continuously capturing an image of the bottle, a Markerassociated with the bottle could remain stationary with respect to thebottle so that it does not move with the movement of the camera.Therefore, the PEPSI brand Object inserted and/or overlaid at the Markerwould likewise appear to remain stationary with respect to the bottle.However, if the User prefers, he could alternatively choose to allow theMarker, and therefore the Object, to move with the movement of thecamera so that the Object remains in view of the Viewer at all timesthat the bottle is also in view.

The User may be able to specify that a single Marker can be insertedand/or overlaid with plurality of Objects, or with multiple types ofObjects concurrently. Such plural insertion/overlay may even allowselection of the plurality of Objects simultaneously. For example, bothan audio and a visual Object (e.g., an .mp3 file and an .mpeg file) canbe inserted and/or overlaid at a Marker. When the Objects are selectedby a Viewer, the image becomes visible to the Viewer, and a music fileor audio message plays at the same time. This is just an exampleaccording to an embodiment, and does not limit the scope or nature ofother embodiments. Alternatively, a Marker may be configured with thiscapability by the Service, and does not need to be, or cannot be alteredby the User. According to an embodiment, a User can place an object at aMarker and lock the Marker to prevent later placement of any otherObject at that Marker. For example, a User may want to associate aViewer-text entry field object at a Marker where Viewers could submitreviews and/or other comments regarding the associated Content, or whereViewers could create a Content-relevant e-mail or text message or othermessage, and send the message to the User or to another Viewer. The Usercould place the corresponding Object at the Marker and set a markerattribute (e.g., marker parameter) to prevent alteration or removal ofthe Object, or placement of any additional Objects at that same Marker.

When the User finishes associating Markers with the Content, or perhapsperiodically while associating Markers, the Service may prompt the Userto make adjustments to Markers. For example, a pattern recognitionsystem provided by a Service may detect something unusual and flag theUser, and may also provide suggested corrective actions available to theUser and relevant to the detected condition. For example, a User mayassociate a bottle-shaped marker with the pattern of a table in a video,and may designate the Marker to remain persistent regardless thecontinued presence of the table pattern. However, in a later portion ofthe video, the table has been removed, and is no longer visible. Apattern recognition system provided by the Service may detect that thebottle, once apparently sitting on the table, now appears to be floatingin mid-air. Therefore, the Service can prompt the User to either removethe Marker (e.g., drag it back to a toolbar), or to move the Marker toanother location within the Content (e.g., onto another table). If theUser removes the Marker, they may opt to remove the Marker from theContent entirely, or from only a portion of the Content.

At 126, the User views the marked Content, checking and adjusting Markerplacement for consistency and artistic appearance as needed. If, forexample, a Marker associated with a stationary object (e.g. a wall) anda Marker associated with a moving object (e.g. a vehicle) conflict dueto a portion of the moving object crossing in front of (or behind) thestationary object, the User can specify an appropriate resolutionaccording to depth priority. For example, the User may be able todesignate the Marker associated with the stationary object as‘Background’, while designating the other Marker as ‘Foreground’. Thiscan, in an embodiment, provide a situation where the moving Markerremains visible and fully selectable when transitioning through theContent image and across (apparently between the Viewer and) thestationary Marker. Meanwhile, the stationary Marker is only selectableduring the transition of the moving Marker to the extent that someportion of it is still visible from behind the moving Marker. Therefore,Markers can be understood as placeable in one or more layers of depthrelative to content and to one another.

During this process, the Service may provide the User with the option tomake all or some portion of the Markers visible to the User for easyreference. The User may be provided with full discretion to placeMarkers wherever and however the User desires within the Content.Alternatively, the Service may provide Marker placement recommendations,or even guidelines (e.g. style guidelines) to which the User mustgenerally and/or strictly adhere.

When the User is satisfied that all Markers in the Content are as theyshould be, the User returns the Content to the Host, at 128. As notedabove, this could mean that the User returns the Content to his ownserver and/or website, or an external server and/or website (e.g.,YouTube).

At 130, the Service may then remove the Content from Service servers,retaining a registry including the unique identification of the Content(and all relevant User-relative data) and all Marker data (e.g., type,location within content, user suggestions for Objectinsertions/overlays). The markers, however, remain embedded in theContent in the locations (e.g. positional and/or chronologicallocations) and manner designated by the User during the associationoperations described above. The Service owns the Markers, and can chargePromoters for access to the Markers for placement of Objects that thePromoters want to deliver to the attention of Viewers of Content.

In much the same ways as described above with regard to Markers, a Usercan alternatively or additionally associate Data Packets with Content.Rather than providing, for example, an insertion and/or overlayplaceholder for later insertion and/or overlay of Objects, a Data Packetmay include a discrete and complete unit of executable content dataplaced relative to the Content. For example, the data comprising a DataPacket may represent, and make available to a Viewer when executed, avideo, text and/or audio clip. Examples include but are not limited toembedding a hidden video, text or audio clip within a video, text oraudio clip—such as a hidden song within the video, a hidden video withinan audio clip wherein the video is selectable when cued by an audioindication and is viewable on a display or other viewable means, ahidden audio-video lecture from an instructor within online coursematerials, etc.), a result of executed software code, a personalmessage, an Internet link (e.g., URL, URN, URI), an n-dimensionalvirtual environment and/or an n-dimensional virtual world that Viewers,readers or listeners can virtually enter into or move throughout, anautomatically dispatched electronic mail message, an opened telephonicconnection or some other type of data encoded content. Therefore, DataPackets can deliver to Viewers of Content many of the same types ofcontent as do Objects (described below). However, the nature and contentof Data Packets generally remain as designated by the User until andunless the User decides to alter, replace, or remove them, or until theyare altered as an operational result of their intended function.

As the examples above should make clear, the ability to associate DataPackets with or relative to Content is not limited to only viewableContent. For example, inasmuch as Data Packets may be self-executing, aData Packet can be associated with purely audio Content, and can executewithout the need for Viewer selection of the Data Packet, or evenwithout Viewer recognition of the presence of the Data Packet prior toexecution. Therefore, Data Packets can be associated with, and providebenefits with, nearly any kind of underlying Content. As with Markerdata, a Service will receive Data Packet data from a User, a Promoter orHost, and retain a record of the Data Packet data includingcharacteristics related to the placement of the Data Packet(s) relativeto the Content.

A Service would make tools available to Users enabling placement of DataPackets relative to Content, and allowing the User to specify how theData Packets are presented to Users that access and view the Content.For example, Data Packets may be embedded into the Content, and may bevisually represented to a Viewer, or may be visually invisible to aViewer. Data Packets can be self-executing, so that the data in thepacket is executed immediately when the Content is viewed, oralternatively, execute at the completion of a predetermined delay (e.g.,predetermined and set by the User) after a Viewer begins viewingContent, or upon Viewer execution. Alternatively, a Data Packet mayexecute and run in a separate window on a User's device display.

Service provided tools may allow control of playback, volume, or otherContent execution relevant parameters pertaining to execution of data inthe Data Packet, separate from and simultaneous with similar or othercontrol of the Content, (e.g., start, stop, pause, resume, volume,speed, repeat, skip, next, previous, undo, edit, send, navigate, recall,save, forward, addressing, block, filter, resize, bookmark, etc.).Execution of data in a Data Packet may also cause concurrent executionof viewable Content to be placed on hold (e.g., paused) until completionof the Data Packet data execution, or until the Viewer intervenes toexercise control over the execution of the Data Packet data, theviewable Content, or both.

A User, Promoter or Host may also provide a Data Packet with encodingwhich allows only authorized Viewers to execute the data in the DataPacket. By this capability, a User, Promoter or Host can deliverinformation to only specific Viewers, although the Data Packetscontaining the information are placed relative to Content which isitself viewable to a broad audience of Viewers. For example, amerchandiser may advertise to all potential customers by making broadlyavailable Content describing his or her products, but may also associatea Data Packet with the Content, the Data Packet including a specialdiscount code for use by preferred customers or ‘members’. Generally, adata packet may extent an invitation to a viewer to engage in acommercial transaction including but not limited to one or more ofcomprising one or more of an announcement regarding a sale, a coupon, arebate offer, a consolidated multi-item purchase price offer, a limitedtime offer, an exclusive offer, a subscription offer, and a productsample, although others are anticipated and fall within the scope of oneor more embodiments.

Codes placed in a Data Packet by a User, Promoter or Host, or insertedat a Marker by a User, Host, Promoter, Viewer, or other can also serveto associate Content With other Content similarly encoded or toassociate and link Viewers, readers or listeners to others on theinternet who are simultaneously engaged in the same/similar internetContent as that of the information or Content placed in the Data Packet.The Service, acting as a network ‘brain’, can establish and maintainassociations between Content related by such codes when an insertedObject/code is selected, or Data Packet code is executed, by a User,Viewer, or other. A Service can also construct and maintain a virtualdirectory of all interrelated Content via codes associated with theContent via Data Packets or Markers. Similarly, codes inserted atMarkers by Viewers, Hosts, Promoters, or others can serve to associatesome or all of these various parties with the Content and can be used tolink them to Users, or to link Users to other Users, Viewers, Hosts orPromoters, and a Service can construct and maintain a virtual directoryof which Viewers, Hosts, Promoters, or others are associated by codesinserted relative to Content. Alternatively, and/or simultaneously,other codes can serve to associate various parties to each other, and/orinto a social network related to the Content, and a Service canconstruct and maintain a virtual directory of which Viewers, Hosts,Promoters, or others have been, are or can be associated and linkedtogether by inserted codes with Content. A Service can monitor and storechat, other means of communications and transactions (such as Contentfile sharing) that occur in environments (windows, areas, n-dimensionalvirtual environments, n-dimensional virtual worlds, etc.) that arecreated by these Data Packets.

The various capabilities for controlling presentation, execution,availability, Viewer control, or other parameters of Data Packets may bedefined by a User when or prior to associating Data Packets withContent. Further, a User can in some embodiments, as a maintenancefunction, change such parameters, or even remove previously placed DataPackets at any time after the Data Packets are associated with Contentand/or after the Content is ‘published’.

In a more ‘open’ embodiment, n-Users can associate Data Packets withContent after publication of the Content by the User. The ability forn-Users to do so could be controlled by the User via a setting enabledat some time between Content creation and publication, or even afterContent is published, as a maintenance capability. In this way, n-Userscan, for example, insert Data Packets which invite subsequent Viewers toconnect with (e.g., automatically, or selectively at Viewer'sdiscretion) the n-User for a social interaction. For example, an n-Usermay wish to gather information from other Content Viewers relative to aproduct or topic presented by the Content. When a subsequent Viewerselects the Data Packet associated with the Content by the n-User, theViewer is invited into an interactive exchange with the n-User (such asa social networking environment, as described herein), or, in anotherexample, may be directed to a web-based survey form placed by the n-Userto gather information. One of ordinary skill in the art will appreciatethe great number of beneficial applications enabled by this and similarembodiments of the invention disclosed herein.

3) Promoter Designation of Objects for Insertion and/or Overlaying atMarkers

Referring to FIG. 2 at 202, Promoters having information that they wishto bring to Viewers' attention can pay the Service for access to Markersin marked Content. Of course, on occasion, the Service may also opt toprovide access to Markers for no charge. For example, Promoters may begiven access to one or more markers for a duration, at the discretion ofthe Service, to demonstrate how the Service can benefit the Promoter. Asanother example, the Service may provide access to Markers above andbeyond what the Promoter pays for as part of a ‘bonus’ to generategoodwill with the Promoter, or as a public service (e.g., when thePromoter is a public service organization).

A Promoter may want to insert/overlay Objects at Markers associated withspecific Content (e.g., a particular video), with specific types ofContent (e.g., video content only), or with Content of a particulartopic or nature (e.g., sporting events). A Promoter could also wish toinsert/overlay Objects with any Markers having User-definedattributes/values that correspond with the Promoter's objectives, orthat are associated with Content the Promoter thinks may be viewed byViewers in his target demographic. The Promoter can specify which ofthese options or combinations of options he wants to use. Of course,other options may be made available by the Service and offered toPromoters.

The Promoter, at 204, will identify Objects to be inserted/overlaid atMarkers. Objects can include brands, products, graphic designs, audiomessages, links to a web page (e.g. URL-defined hyperlink),n-dimensional virtual environments and/or worlds, other Content,downloadable material (e.g. software), surveys, order forms, or email.This list is not exclusive, however, and many other options as will beapparent to those having ordinary skill in the art. For example, aPromoter may want a brand name (e.g., ‘Jam Juice’ Energy Drink) toappear in Content relating to sporting events.

Objects may also simply consist of associations. Selection of anassociation Object can convey a Viewer to Object-linked information,such as to a website, an-dimensional virtual environment and/or world,or Content residing on a server. Association Objects can also be used toconvey Viewers to a utility, such as an email utility, a survey utility,a job application utility, or countless others as will be apparent tothose of skill in the art. Therefore, Objects are not limited to text,graphics, or other visible images, but may include associations throughwhich a Viewer can be linked with nearly any kind of information.

The Promoter can then provide and/or define Object-relevant values andlink them with an Object, at 206. Object relevant values may describevisual attributes of the Object (e.g., size, shape, color), generalcontextual attributes (e.g., Object is related to sports or to aspecific sports activity), specific contextual attributes (e.g., theObject is a bottled beverage), or any number of other attributes,properties, themes, qualities, or descriptive terms that can aid inrelating Objects to Markers and/or Content to meet a Promoter'sobjectives. Promoters can provide/define Object relevant values whichrelate an Object to specific Markers, or to Markers related to aspecific Object, brand, or type of Objects by User suggestion (e.g., asat 124 in FIG. 1 b), or to markers in specific or general classes ofContent.

Alternatively, a Promoter may desire for the insertion and/or overlay ofspecific Objects at specific Markers and/or in specific Content. TheObject-relevant values may include, or may solely consist of suchinstructions. For example, the Promoter may also be a User, and mayprovide Content via a Host and associate Markers with the Content. Inthis situation, the Promoter may have a plan for promoting products toViewers, and will therefore designate Objects for insertion/overlay atMarkers in the Content according to that plan. In another instance, amovie studio as Promoter may want to promote new release movies.Therefore, the studio may create or cause the creation of Content, andassociate Markers with the Content, into which Markers areinserted/overlaid Objects consisting of movie posters or links to Moviepreviews. These are just a few examples, and do not in any way limit thescope or nature of the nearly unlimited possible embodiments of theinvention.

Once the Promoter has provided and/or defined Object-relevant values forone or more Objects, the Promoter then provides the Service with accessto the Objects, at 208. Providing access may include providing theObject to the Service for storage on the Service's server(s), orproviding the Service with access to the Objects on the Promoter'sserver(s). The Objects may also be maintained somewhere other than on aserver residing at the Service or the Promoter, but may be maintained ata separate location where the Objects are accessible to the Service, thePromoter, or both.

In embodiments of the invention, ‘access’ may include actual possessionof an original or copy of the Object on a Service server or a form ofstorage medium compatible with the form of the Object (e.g., opticaldisc, magnetic storage media, photograph, a list). ‘Access’ may alsoinclude constructive possession of an Object through permission andcapability to access an original or copy of the Object from a serverowned by someone or something other than the Service. Therefore,‘providing access’ can include informing the Service of the location ofthe Object, such as a publicly or commercially available archive orother source, even if the Service still must obtain rights to use theObject. Providing access may involve physically conveying storage. mediacontaining the Object, uploading or otherwise transferring the Objectvia an electronic link, or numerous other methods. In embodimentswherein the identities of the User, the Host, and the Promoter partiallyor fully overlap, the actions involved in or required for providing theService with access to Objects may be substantially streamlined.

4) Activation of Object Insertion/Overlay

Referring to FIG. 3 at 302, a Viewer accesses a Host, and selects markedContent from the Host for Viewing. The Content may be activated andviewed at the Host website (e.g., YouTube) using tools and capabilitiesprovided to Viewers by or through the Host, or may be downloaded fromthe Host to the Viewer for later viewing. In the latter situation, theContent may be considered ‘selected for viewing’ when the downloadedContent is later opened (e.g., a Content file is ‘run’ so that theContent can be viewed) by the Viewer, rather than when the Content fileis downloaded.

At 304, the Service detects the Viewer's selection of marked Content. Insituations where a Viewer selects the Content for viewing at the Hostwebsite, the Viewer generally is already connected to the internet. Inthe situations where a Viewer selects Content for viewing afterdownloading a Content file from the Host, the Viewer may or may notstill be connected to the internet. If the Viewer is not connected tothe Internet, selecting the Content for viewing will cause the Contentto initiate/open an Internet . connection so that the Service can detectthe Viewer's selection of the. Content for Viewing. Alternatively, ifconnection to the internet is not currently available, selecting theContent will establish a queue. At the next subsequent availability ofan internet connection (e.g., the Viewer reconnects the device to awired, wireless, or other internet-linked system), the Viewer's devicewill establish a connection to the Internet and the Service will detectContent selections held in the queue: Alternatively, execution ofContent selections in a queue can be scheduled, so that an internetconnection is established on a schedule preset by the Viewer, ratherthan automatically when such connection is available.

At 306, the Service will recognize the unique Content identificationassociated with and/or embedded into the Content at the time the Userregistered the Content with the Service. Once the Service identifies theContent, the Service can then select Objects, at 308, to insert and/oroverlay at the Markers in the Content. The Service may spontaneouslyselect markers based on keyword/value relationships as provided by Users(relating to Markers) and by Promoters (relating to Objects), or mayselect Objects based on more firmly prescribed instructions linkingspecific (or relatively specific) Markers with specific (or relativelyspecific) Objects, as described above.

At 310, the Service then inserts and/or overlays the Objects at theMarkers in the Content, and at 312, transmits the Content with theMarkers and inserted/overlaid Objects to the Viewer for viewing.Transmitting the Content may simply involve causing the Content to beginplaying on the Host site or the Viewer's display (e.g., commencing theplaying of a video, game, or other non-static content, entry into ann-dimensional virtual environment and/or world), or causing a Contentimage to appear on the Viewer's display (e.g. by displaying aphotographic image, computer generated image, or causing a computer torender an image real-time), although the embodiments are not so limited.Because Markers can exist in a separate content layer overlying markedContent, Objects can be inserted at a Marker in the separate contentlayer, and simultaneously overly the marked Content. In an embodiment,therefore, an Object can be both inserted and overlaid relative toContent.

The Objects may all be inserted/overlaid into. the Content prior to theContent becoming viewable by the Viewer. Alternatively, Objects may beinserted/overlaid progressively as the Viewer continues to view theContent. This latter situation may be particularly true when the Contentis of an interactive nature, for example a computer game or ann-dimensional virtual environment and/or world. In such situations, theViewer's actions throughout the game can cause marked portions ofContent to appear relatively randomly (e.g., not according to apre-determined schedule or invariable sequence). Therefore, Objects willbe inserted/overlaid ‘on the fly’, as the game or other activityproceeds, rather than entirely prior to commencement. This may mean thatObjects are continuously downloaded and inserted/overlaid real time bythe Service throughout game play or other activities, or it may meanthat the Service downloads all Objects that may be inserted/overlaid atMarkers in the Content prior to commencement of the game or otheractivity, and the objects are then inserted/overlaid by the Service asthe game or other activity proceeds. If a single Marker appearsrepeatedly throughout the viewed Content, either the same or differentObjects can be inserted at the Marker during each subsequent appearance.

As mentioned, Content may be selected and downloaded for later viewing,and later selection, of Objects. Similarly, Data Packets in Content canalso be saved for later access by a Viewer. Data Packets may containinformation for which a connection to the internet is necessary, or maybe fully executable without an internet connection. In those situationswhen an internet connection is necessary (e.g., Data Packets which, whenexecuted, navigate to a website or other web-based presence, or act toacquire remote information via the internet), the Data Packets may behandled similarly to Content with Markers and inserted/overlaid Objects.Later selection/access of the Data Packet(s) can either cause aninternet connection to be established, or, alternatively, a queue may beestablished, or added to, so that the Data Packet(s) can execute via alater internet connection. Data Packets not requiring an internetconnection can be saved on the Viewer's device, or on a peripheralstorage device, and may be accessed at either a scheduled time or at theViewer's leisure. Alternatively, a Data Packet may self-execute at alater time based upon a time-based attribute of the information in theData Packet, pre-set by the User, the Viewer, or a third-party.

5) Linking Viewers in Social Networks via Analysis of Metadata of theMarkers, the Objects and the Content.

The Service can perform an analysis of the metadata description data ofMarkers, Objects and Content to link Viewers of the same and/or similarMarkers, Objects and/or Content to each other, in the same or a similarmanner as is described in U.S. Provisional Patent Application Ser. No.60/856,404, entitled USING INTERNET CONTENT AS THE MEANS TO ESTABLISHSOCIAL NETWORKS BY LINKING INTERNET USERS, SEARCHERS, VIEWERS AND/ORLISTENERS OF THE SAME AND/OR SIMILAR INTERNET CONTENT TO EACH OTHER VIAA PLATFORM THAT ENABLES ANY OF THE FOLLOWING IN N-DIMENSIONALENVIRONMENTS: SOCIAL NETWORKING, COMMUNICATIONS, SHARING, CO-BROWSING,E-COMMERCE, ADVERTISING, SEARCH, HOSTING AND REGISTRY SERVICES, PUSH ANDPULL APPLICATIONS, ANONYMOUS COMMUNICATIONS, AND RICH PRESENCE, filedNov. 2, 2006.

As shown at 1602 in FIG. 16, User files with the Service Metadatadescriptions for types of Objects to be placed in Markers that the Userplaces in Content.

As shown at 1604, when a Viewer accesses Content that contains Markers,the Service downloads relevant Objects into the Content for that Viewer(based on Viewer attributes/interests). In addition, and perhapsconcurrently in some embodiments, the Service also analyzes the Metadatadescriptors of the Markers and searches its database of Viewers (e.g.,who are currently online or who have left instructions to be laterconnected into a live social network in which another online Viewer isengaged in Content having the same or similar Metadata descriptors asthe Markers). The Service matches and links the Viewer to other Viewers(who are online at that moment), and who are viewing Content having thesame and/or similar Metadata descriptors in the Markers in the Contentthey are viewing.

As detailed at 1606, the linking of Viewers who share a commonality ofinterests (as defined by the same and/or similar Metadata of the Markersthat are inserted and/or overlaid into Content), enables new forms ofSocial Networking, Co-Browsing, Co-Shopping, E-Commerce, Advertising,Customer Service, and other beneficial activities.

In an alternative embodiment, a User can insert a code as a Data Packetassociated with Content, and also associate a Marker(s) which allowsViewers to insert and/or overlay a code at the Marker(s). By thismethod, the Users and/or Viewers can signify their intent (e.g.,availability, willingness, desire) to interact (e.g., live or otherwise)with other Users and/or Viewers with regard to the Content. Insertion ofa code by a Viewer can act to allow the Viewer to be linked into aninteractive social environment, either immediately or upon approval bythe User or a moderator, and can notify the User and/or other Viewersthat the Viewer is entering the social environment, whether or not theUser and/or other Viewers are currently interacting in the socialenvironment. Likewise, inserting their code at a Marker may enablenotification of the Viewer when the User or other Viewers (or moderator,or others) have entered the social network and are available for live orother interaction, or upon the occurrence of some other event related tothe Content, such as the revision or replacement of the Content, or thepublication of associated Content.

By the above described code insertion capabilities, Viewers (as well asUsers, Promoters, or others) can be linked together to enable concurrent(e.g. live) and/or later viewing of Content. During such simultaneous orlater viewing (which may alternatively involve listening to audiocontent, watching visual content, or otherwise perceiving other or mixedmedia Content), Viewers can concurrently interact with each other and/orwith the Content in real time through a concurrent or otherwiseinteractive environment. For example, while geographically separated, adirector and a cinematographer could simultaneously view a sequence offilm with audio and/or video components, and discuss edits to be made tothe film. Using tools provided by a Service, one or both of them couldact as a User or n-User, and insert Markers at various locations in theContent film, at which either of them could later insert instructions asObjects for later access by film editors. Alternatively, instructionscould be associated with the Content film as Data Packets.

In another embodiment, two or more Viewers can be linked for concurrentviewing of Content although the Viewers are not otherwise acquainted.When one of the Viewers selects the Content for viewing, a Servicerecognizes codes previously inserted at a Marker by other Viewers (or aUser, or Promoter, or others) associated with the Content, and links theViewer with one or more of the other Viewers, etc., for example in alive social network with each other. Therefore, the Content can beconcurrently experienced by multiple viewers, who can also interactrelative the Content via an interaction-enabling means (e.g., text,audio, or other communication means and/or methods) described herein orotherwise known to those having skill in the art.

In another embodiment, two or more Viewers can be linked forsimultaneous viewing of Content although the Viewers are not otherwiseacquainted. When one of the Viewers selects the Content for viewing, aService recognizes codes previously inserted at a Marker by otherViewers (or a User, or Promoter, or others) associated with the Content,and links the Viewer with one or more of the other Viewers, etc., as ina live social network with each other in an n-dimensional environment.Therefore, the Content can be simultaneously experienced by multipleviewers, who can also interact relative the Content via text, audio, orother communication methods described herein or otherwise known to thosehaving skill in the art.

Objects that are inserted/overlaid into Content can be used for thepurpose of collecting specific information that is being generated,exchanged, viewed, listened to, or interacted with within a live socialnetwork and/or within an n-dimensional Virtual Environment. Informationgathered from an inserted/overlaid Object can then be used to create anew live social network and/or n-dimensional Virtual Environment. Forinstance, the Object that has been inserted/overlaid into the Contentcould be the image of a CD that, when clicked on, can record onto CDmedia the audio content exchanged within the live social network and/orwithin the n-dimensional environment. In an embodiment, clicking orselecting by some other means, the image of the CD object that has beeninserted/overlaid into the Content, or selection of some otherviewer-perceivable indication of code, can launch an analysis of themelody of audio content (or more generally, can analyze the qualitativeproperties of any other type of content being perceived by a Viewer)that can then be used to create a live social network and/or ann-dimensional Virtual Environment, as substantially depicted anddescribed in FIGS. 17 a-17 s.

The object that is inserted/overlaid by a Viewer into the Content couldbe the image of a DVD that can record onto DVD media the audio and/orthe video Content exchanged within the live social network or within then-dimensional environment. In an embodiment, by clicking or selecting bysome other means the image of the DVD object inserted/overlaid into theContent, could launch an analysis of the melody of the audio and/or asearch for specific images (using image recognition techniques) tocreate a live social network and/or an n-dimensional Virtual Environmentfrom that information that is captured.

A Radio Broadcast object or a TV Broadcast object, that has beeninserted/overlaid into the Content, can when clicked on, broadcastwithin a live social network and/or within a n-dimensional virtualenvironment, and/or through the internet to other on-line sites and/orapplications and/or to off-line destinations, the content and/orcommunications gathered or originating within the live social networkand/or the n-dimensional Virtual Environment. By clicking on orotherwise selecting a CD/DVD/Radio/TV Object or any other Objects thatare inserted/overlaid within the Content, live social networks andn-dimensional environments can be created by the means described in thisinvention.

Live Content exchanges between Users, in live social networks and/or inn-dimensional Virtual Environments that have been created from Objectsinserted/overlaid relative to Content, could become live Content channelbroadcasts (Audio, Video, Audio-Video) to other Viewers in other onlineand/or offline communities. Stored historical information can bebroadcast contemporaneously or later as either live or on-demand channelbroadcasts.

6) Viewer Selection of Objects

As shown at 402 in FIG. 4, while viewing Content into which one or moreObjects are inserted and/or overlaid, the Viewer can select an Object.An Object can be indicated and selected by any device or systemordinarily or specially available to computer users for indicatingand/or selecting items or regions of a computer display screen or thedisplay screen of any other electronic device. For example, a Viewercould select an Object by using a computer mouse, keyboard or keypad,gaze tracking system, sip and puff control system, or joystick, althoughthis not an exclusive list and the embodiments are not to be understoodas so limited. If the Viewer views the Content on a touch sensitive orlight sensitive display screen, the Viewer can select an Object using astylus, their finger, a laser-pointer, or some other device or method bywhich the display recognizes the selection and associates it with theObject. As shown at 1406 in FIG. 14 b, the Viewer can place a cursor orother indicator on, at, over, within, or otherwise corresponding to aportion of an image in the Content, and act to select an Objectcorresponding to that portion of the image. The Object itself may bevisually perceivable by the Viewer, or its presence may be detectableonly by a Viewer detectable change in the cursor or other indicator. Inembodiments, simply placing an indicator (e.g., cursor) on, at, over,within, or otherwise corresponding to an Object is sufficient to selectthe Object.

Alternately, moving an indicator over an Object can cause a message orother indicator to appear somewhere in a visible portion of the displaydevice being used by the Viewer, indicating the presence of an Object.Likewise, moving an indicator over an Object could generate an audio orother detectible cue to the Viewer that they can select an Object. Forvision impaired users, an audio cue can also rise and/or fall in tone,volume, or frequency as the Viewer causes an indicator to increase ordecrease in proximity to an Object.

At 404, the Service detects the selection and identifies the Object. Inembodiments, this may entail the Service detecting the selection in realtime. In other embodiments, tools and/or capabilities provided by theService to a Viewer, Promoter, User, Host, third-party, or to a deviceor system controlled and/or operated by one or more of these entities,and capable of responding as provided for by the Service, may detect theselection in real time as a proxy for the Service.

In one embodiment, shown at 406, the Object selection is stored so thatit can be accessed and reviewed by the Viewer at a later time. Theselection may be stored on a server or some other storage device ormedium at the Service, or it may be stored by any resident data-storagemeans (e.g., RAM, hard drive, optical storage medium, USB connectstorage device) of or linked to the device by which the Viewer accessedand/or viewed the Content. Thus, at some later time, the Viewer canaccess stored Object selections, as at 408, and review the datarepresented by those Objects. This allows a Viewer to continue theContent viewing experience relatively uninterrupted by their selections.This may be particularly advantageous to Viewers that viewtiming-sensitive Content, such as interactive computer based gamesinvolving other players, or who otherwise only have limited time to viewthe Content.

In another embodiment, shown at 412, the Object selection concurrentlylinks the Viewer in real time to other n-Viewers who are concurrentlyengaged in the same/similar data or context associated with the Object,thereby creating a live social network among these linked Viewers.

Alternatively, as shown at 410, the data associated with a selectedObject can be reviewed by the Viewer substantially immediately. In suchinstances, when warranted by the nature of the data linked to an Object(e.g. viewable data), another window may open on the Viewer's displayscreen to display the linked data (e.g., a web page, picture, video). Inthe case of non-static Content, immediate review of the Object-linkeddata may ‘pause’ the Content so that the Viewer does not miss any of theContent while reviewing the linked data. Of course, the option to pauseor not pause the Content during review of Object-linked data could alsobe provided as a Viewer selectable option. In another instancepertaining to immediately viewable Object-linked data, selection of anObject can terminate viewing of the Content and begin review of theObject-linked data. This also may be configured as a Viewer selectableoption.

Object-linked data may be considered ‘substantially immediatelyreviewable’ for at least the reasons that delays may be experiencedbetween the time that a Viewer selects an Object, and the time that theObject-linked data is presented to the viewer. Delays may result fromtransmission, buffering, image rendering, hardware and/or softwarelimitations, or other similar causes. However, for the purposes of theembodiments described herein, Object-linked data may be considered‘immediately reviewable’ any time an Object is selected and the Objectis not stored for later review.

FIGS. 14 a-g show embodiments wherein Object-linked data is immediatelyviewable by a Viewer. With reference to FIG. 14 a, a Viewer viewsContent 1402, in this example a skateboarding video, using a computerprogram 1404 (e.g., Windows Media Player) adapted for displaying theContent. In FIG. 14 b the Viewer places an indicator 1406 within theselectable portion of an Object 1408 (e.g., the image of askateboarder's shirt), and a defined boundary 1410 corresponding to aMarker becomes visible to the Viewer (e.g., a line drawn using atelestrator-like tool or capability). Because an Object isinserted/overlaid at the Marker, the Viewer can select the Object, andObject-linked informational content 1412 (e.g., clothing merchandise)becomes Viewable by the Viewer in a separate area proximate to theContent 1402. Likewise, as shown in FIG. 14 c, the Viewer can move theindicator to a portion of the Content image corresponding to anotherMarker and Object 1414, associated in this example with a skateboarder'skneepad. When selected, other Object-linked content (e.g., skateboardingequipment) is immediately viewable in a separate area of the devicedisplay adjacent to the Content. The Object-linked content typically butnot exclusively includes Promoter-relevant items, such as items (e.g.merchandise) the promoter offers for sale to viewers. Therefore, theViewer can typically select one or more of the Promoter-relevant itemsdisplayed in response to the Viewer's selection Of an Object 1414.Further, the items themselves can likewise be selectable. Selecting anitem can initiate a commercial transaction allowing the Viewer topurchase merchandise, can initiate a communication means (e.g., e-mail,telephonic, instant message session) with the Promoter or another, canactivate a virtual environment in which, or example, an avatar of theviewer can try on a t-shirt, or can link the Viewer (e.g., live in realtime) to other Viewers based on metadata associated with the item,although the embodiments are not so limited.

In FIG. 14 c, the selectable Object is inserted at the Marker, and iseffectively invisible, in that the Content image (e.g., the kneepad) isvisible to the Viewer, and is not obscured from View by an overlaidvisible Object. However, as shown in FIG. 14 d, an Object can beoverlaid at a Marker so that it is visible to the Viewer. Here, thevisible Object 1418, a logo for a product (e.g., ‘Jam Juice’) isoverlaid at a Marker associated with a portion of the Content imagecorresponding to a flat surface 1420 (e.g., a wall). The visible Object1418 overlays portions of the flat surface, obscuring those portionsfrom the Viewer's view. Once again, moving an indicator 1406 over theObject 1418 and selecting the Object 1418 causes Object-linked content1416 (also shown in FIG. 14 c) to be presented in an area of the displayscreen proximate to the Content 1402.

With reference again to FIG. 14 b, the image of the skateboarder's shirtmay appear in the original or marked Content substantially as shown inFIG. 14 b, and the Object 1408 may be inserted so that it issubstantially or completely invisible to the Viewer when viewing theContent. However, it is also possible that the skateboarder's shirt wasanother color (e.g., white) in the original Content, and the Userassigned color to the Marker while or after associating the Marker withthe portion of the Content corresponding with the skateboarder's shirt.Therefore, whenever and wherever the shirt is visible in the Content,all or a portion of the shirt appears to be the color assigned to theMarker by the User.

Likewise, in embodiments, a visible Object corresponding closely to animage in Content, such as the skateboarder's shirt and the Object-linkedmerchandise 1412 shown in FIG. 14 b, can be overlaid at the Marker. Thismay be done by matching values, keywords, or other Content and/or Objectrelevant information (e.g. metadata descriptive information) provided byUsers and/or Promoters, respectively. Therefore, throughout the viewingof the Content, the visible Object overlays and visually replaces aportion of the viewable Content (e.g., the skateboarder appears to‘wear’ the overlaid Object shirt).

In FIG. 14 e, the image of a ‘coupon’ Marker 1422 on an Object indicatesthe presence of an overlaid ‘coupon’ Marker. When the indicator 1406 iseither rolled over the ‘coupon’ Marker 1422 or the ‘coupon’ Marker 1422is clicked on, coupon Objects 1426 appear within the Content 1402 or atother areas of the display. The Server could have instructions for“coupon ads”, configured as space that vendors could pay to have anadvertisement placed within, such as by placing bids and being awardedaccess to the space upon placing the highest bid. In one embodiment, thecoupon ads could be inserted within a “book” in a room or in an areathat is visible to the Viewer so they can be viewed within one locationthat is understood to be for coupon ads. The Service could provide moreareas for those who want to “pay” for inclusion in other parts of thevisible areas—e.g., a branded Object that would have an indicator on itthat there's a coupon available for it. The branded Object could serveas advertising for the brand and the Coupon Indicator (pictured here asa Star icon) could serve as a means to access coupons.

In FIG. 14 f, we see that by clicking onto areas of the Content 1402, aViewer can elect to view and/or enter an n-dimensional virtualenvironment or world 1428, such as to be linked in a social network withother Viewers who are concurrently engaged in the same and/or similarContent. For example, FIG. 14F shows several Viewers who have been putinto an n-dimensional environment or world together and are viewing thesame Content, and these Viewers can communicate with each other, shareand exchange Content files, and engage in other mutual/sharedactivities.

In FIG. 14 g, coupon Markers (as indicated by the star 1422) can beplaced on or within Content in an n-dimensional environment and/or worldto launch the opening of coupon Objects 1426 that can be visiblyoverlaid within the Content or elsewhere in the display.

The examples described above should not be taken to imply that insertedObjects are necessarily invisible while viewing Content, nor thatoverlaid objects are necessarily visible while viewing Content. Thevisual attributes of visibility or invisibility can be assigned to bothinserted and overlaid Objects without any necessary limitation of eitherattribute to either type of Object placement. Likewise, Objects bothvisible and invisible may be inserted or overlaid at Markers, whetherthose Markers are visible or invisible. In embodiments, a visible Objectmay obscure a portion of a visible Marker.

In the case of Data Packets associated with Content, the Data Packetsmay be either Viewer selectable or automatic (e.g. self-executing), ormay be extractable through a software program that is designed to filterand extract data within Data Packets. With regard to Viewerselectability, selection of a Data Packet can result in any of a numberof outcomes. For example, Viewer selection of a Data Packet may causethe data therein to execute, such as to play a video and/or audio clip,to display a message, to link to a webpage, or to cause some other Userintended result. Alternatively, selection of a Data Packet by a Viewermay cause the Data Packet to be stored at or on a computer readablestorage medium (e.g. a hard disc drive, a USB flash memory drive).

The data in a Data Packet could include links to web pages or sharedwindow experiences with the Data Packet originator(s), so others cancommunicate with those who have retrieved certain codes or data from theData Packets. For example, and to keep the Data Packets from being solarge that they slow down the Content file downloads, the Data Packetinformation could be linked to external servers, web pages, systems orother entities that host web pages, and/or social networking sites. Suchentities could keep records of all comments, correspondence or postsmade into the Content, for review by all or by certain Data Packetretrievers. In an embodiment, a hosting entity could return and open ina separate window, or overlay on top of or associated with the Content,the messages that people want to leave and share with others who areviewing that Content. These messages/posts could be advertisements,communications among friends, invitations to join social networkinggroups, or other communication generated at least in part by exposure toContent, Data Packet information, and/or Objects. In another embodiment,the information within a Data Packet can be used to concurrently linkthe Viewer with one or more others who are engaged in the same and/orsimilar metadata as the Data Packet, or of Content in a live, sociallink, to communicate and share with those other Viewer(s).

As discussed, a User can designate what action is taken upon Viewerselection of a Data Packet, or a Viewer can so designate. In a preferredembodiment, the Viewer's designation will have priority over a User'sdesignation, although in other embodiments a User's designation maycontrol. Alternatively, the Service can override a designation of aUser, a Host, a Promoter, or perhaps even a Viewer, such as to resolve aconflict, to prevent damage (such as by viruses or other sources ofpotential device or data corruption), to secure confidentialinformation, to update Service tools, or for some other purpose.

Viewer selection of a Data Packet can also cause a confirmation messageto be sent to the originating User, confirming that the Viewer hasreceived and selected the Data Packet. According to this embodiment, aData Packet can provide a valuable ‘return receipt confirmation’function, similar to what is available in tangible postal systems (e.g.,United States Postal Service) and/or email systems, even withoututilizing these other systems.

In yet another embodiment, upon selecting Content for viewing, a new‘window’ may open on a Viewer's device display, containing a directory(e.g., menu, index) of all Data Packets, and/or Objects associated withthe Content. Display of a ‘directory window’ allows the Viewer to selectData Packets and/or Objects individually or in combinations. Soselected, the Viewer may be able to designate and initiate (or scheduleinitiation of) a desired activity, such as viewing, storing, responding,navigating to, forwarding, or other options, with respect to the DataPackets and/or Objects. When selecting multiple Data Packets or Objects,a Viewer may choose to initiate an action with respect to each accordingto a sequence, as a batched activity, or according to some otherschedule. For example, a Viewer may select one or more Objects and/orData Packets from a directory and inspect them for viruses in a batchprocess using a computer virus scanning software program, prior totaking any other actions (e.g., executing, storing, forwarding) withrespect to any of them.

Data Packets may also be associated with Content to provide additionalinformation related to the Content itself when selected by a Viewer. Forexample, a Data Packet can display, in a separate window or as textoverlying some portion of the Content itself, information about theactors, setting, director, or some other aspect of viewable content,such as a short movie clip. Alternatively, a Data Packet can provideinformation about various products depicted in viewable content,allowing a Viewer to navigate to a website where the User can purchasethe products.

Data Packets can function like ‘sticky notes’ placed in Content. AViewer can select a Data Packet, or a Data Packet may automaticallyactivate when a portion of active Content plays, and the Data Packet maycause a message to appear, remain temporarily, and then disappear onceagain from view. Therefore, Content can be used as a type of messageboard and/or reminder tool for Viewers, through the use of ‘pop-up’messages delivered by Data Packets associated with the Content (e.g., as‘notes’ posted for their friends to see). Alternatively, Data Packetsallowing subsequent n-User data entry can function as a ‘visitorregister’ for Content Viewers to record their names and/or commentsregarding the Content.

Data Packets can be particularly useful for providing confirmation ofthe occurrence of an activity or affirmation. For example, distancelearning may be accomplished by making available to a remote Viewercertain educational content. One or more Data Packets may be associatedwith the Content by a User, such as an educational provider, wherein thepackets monitor the Viewer's progress through Content and provideconfirmation to the User that a Viewer viewed the Content.Alternatively, the Viewer may be prompted by automated ‘reminder’ DataPackets to activate other Data Packets at the end of a Content section,the activation of which provides affirmation by the Viewer that theyhave viewed the Content. Data Packets can also deliver quizzes tostudents regarding material viewed by the student, to monitor and ensurestudent comprehension. By these and/or similar embodiments, providers ofdistance learning can remotely deliver and confirm students performanceof coursework, and other Users can achieve similar ends with regard toaudiences for their Content.

Conversely, students, researchers, and others can create content (e.g.,research papers, reports) combining multiple media forms. For example, astudent can create a ‘base’ Content (e.g. a text report), and associatewith the content, at various locations throughout, Data Packetsproviding notes, video clips, photographs, links to websites,invitations to 3-dimensional interactive exhibits, inquiry submissionforms, or other informative materials. Therefore, a rich and activeinformational experience emerges from an otherwise textual reportthrough the use of associated Data Packets.

Although it is not repeated with regard to each and every embodimentherein, Data Packets associated with Content can also collect and ormonitor information regarding Viewers' activities, and distribute thatinformation to designated entities (e.g., the Service, the User, a Host,a Promoter, or a third party such as viewer's parent or teacher, amongothers). This may be true with regard to any or all of the embodimentsdescribed herein. Data Packets can also prevent a Viewer from accessingcertain Content without first viewing other Content and/or taking someaction. For example, a Viewer may not be able to view Content withoutfirst viewing a licensing agreement or a copyright warning, and/oracknowledging having done so.

7) Viewer Attribute-based Insertion/Overlay

As shown in FIG. 5, embodiments of the invention also provide forinsertion/overlay of Objects at Markers based upon various attributes ofa Viewer. These and related embodiments provide a high degree offlexibility and effectiveness to, and increase the revenue generatingpotential and efficiency of applications of the embodiments.

In general, a Viewer accesses a Host, at 502, and selects marked Contentfor viewing. As previously described, the Service can detect theselection of marked Content, and can identify the Content by its uniqueidentification associated and/or embedded into the Content duringregistration by the User. However, according to an embodiment, theService also recognizes, at 504, the geographic location of the Viewer.For example, the Service may recognize that a Viewer selecting Contentfor Viewing is located in Japan. Therefore, at 506, the Service is ableto insert and/or overlay Objects of relevance to Viewers in Japan atMarkers in the selected Content. Further, the Service may haveagreements with Promoters of products, services, or information that thePromoter wishes to direct toward Viewers in Japan, whether thosePromoters are located in Japan or not. By this embodiment, Promoters cantarget information to specific (e.g., cities, provinces) or general(e.g., countries, regions) geographical areas without the need to createwholly separate Content for each.

In another embodiment shown at 508, the Service recognizesViewer-specific attributes, and inserts Viewer-relevant Objects atMarkers in the selected Content, at 510. In one instance, the Viewer mayhave provided to the Service information about the Viewer's interestsand/or other demographic information pertinent to the Viewer or to agroup with which the Viewer is associated. This information may havebeen provided by the Viewer via an on-line Viewer information survey orother data-collection means available from the Service. This information(the Viewer-specific attributes that are provided to the Service) andthe Viewer-relevant Objects that are inserted into the Markers, could beused by the Service to link Viewers to other Viewers to create socialnetworks around the commonality of their interests, in the same or asimilar manner as that described in U.S. Provisional Patent ApplicationSer. No. 60/856,404, indicated above.

The Viewer, while performing internet searches for content may specifykeywords, for example ‘snowboard’, to guide search engines (e.g.,Google, Ask.com, Yahoo) during a search. Such keywords can serve asViewer attributes indicating topics of interest that the Service can useto insert/overlay Objects of interest to the Viewer. As in the examplegiven, the Service can insert/overlay Objects related to snowboardinggear, suppliers, and/or facilities, just to name a few possibilities.

In another instance, the Viewer may have provided such information tothe Host as part of a registration process for access to the Host, andthe provided information is made available to the Service. Due toon-line privacy concerns, such information frequently may be providedonly with the Viewer's consent. Absent specific consent, Viewerinformation may be provided to the Service by a Host in cumulative form,relating to viewers similar in type to the Viewer, without beingspecific to or identifying that individual Viewer. Alternatively, theViewer's anonymity can be preserved by the use of neutral tokens asdescribed in Provisional Patent Application No. 60/856,404, indicatedabove. In still another instance, the Viewer may be given theopportunity to provide information prior to, while, or after viewing theContent. This information can then be provided to the Service and theService can update the Objects inserted/overlaid in the marked Content,or use the information to select appropriate Objects for inserting insubsequently selected Content.

The Service may also have the tools and/or capability to track aViewer's on-line habits, such as which sites they visit, what kinds ofmusic or other items they download, from which on-line retailers theymake purchases, and other such information which can also constitute‘Viewer Attributes’. By such information, the Service can insert/overlayObjects that are not only relevant to a Viewer's interests, but whichcan also change as a Viewer's interests, as indicated by their habits,also change. in addition, the Service can use these “Viewer Attributes”to link Viewers to other Viewers to create social networks around thecommonality of their interests, in the same manner that is described inU.S. Provisional Patent Application Ser. No. 60/856,404, indicatedabove. This process follows the same or similar steps as in theembodiment depicted in Figure 16.

In still another embodiment, the Service can recognize Viewer and/orPromoter time-relevant attributes at 512, and insert time-relevantObjects at Markers in Content at 514. For example, a Promoter may selectand specify an Object for insertion/overlay only after a specifieddate/time. Therefore, the Object would not be inserted until after thespecified date and/or time. Likewise, a Promoter may specify a timeand/or date after which an Object should be inserted/overlaid, or somecombination of start/end dates or times, as well as a duration for andan interval between each. In such situations, the Service can detect thetime and/or date when a Viewer selects Content for viewing. When suchtime/date falls within a time/date specified for inserting/overlaying(or ceasing to insert/overlay) the time-specified Object, the Servicecan insert/overlay (or cease to insert/overlay) the Object(s) asrequested by the Promoter.

The ability to recognize time attributes enables Promoters to presenttime-sensitive information to Viewers. For example, clothing retailerswho schedule specific fashions or seasonal clothing promotions tocoincide with the specific times of the year can provide Viewers withaccess to information about those products at the appropriate times.This can include notifying Viewers of short-duration sales,limited-availability items, special events, or other time-sensitive ortime-limited information.

Likewise, Viewer-relevant time attributes could relate to the seasonand/or time of date at the Viewer's location, enabling a Promoter toprovide information relevant to the Viewer's time-based needs. In anembodiment, a Viewer may be able to set ‘alarms’ or other time ordate-relevant indicators, such as for birthdays or anniversaries offriends or family members. A Service could then detect those indicators,and insert and/or overlay marked Content Objects relevant to theViewer's time-based needs. For example, Objects could includerecommended gift items, coupons, personal messages, reminders, as just afew examples.

Of course, a Service could likewise recognize both Viewer and Promotertime-relevant attributes, and meaningfully related them to provideincreased benefit to both the Viewer and the Promoter. For example, aPromoter may wish to schedule the release of information to coincidewith time-based events at other locations. The information could be atelevision broadcast of New Year's Eve events in New York's TimesSquare, to be delivered to each Viewer time zone only after midnight inthe time zone. Therefore, the Promoter could associate time-relevantvalues with an Object (e.g., link to a broadcast), and a service willonly insert and/or overlay the Object when a Viewer selects Content forviewing and the Service detects that it is after midnight where theViewer is located, based on, for example, a clock on the Viewer'scomputer or other content viewing device.

According to yet another embodiment, the Service recognizes attributesof the selected Content at 516, and at 518 inserts/overlays Objectshaving relevance to the Content attributes at Markers in the Content.For example, Content-relevant attributes can be values attributed to theContent by the User during Content registration. As described earlier,recognizing Content-relevant attributes allows the Service toinsert/overlay sports-related Objects at Markers associated withsports-related Content. This increases the relevance ofinserted/overlaid Objects with the underlying Content, and increases thechance that Promoters can direct information more effectively at Viewerswith relevant interests. That is, Viewers of sports-related Content maybe more likely to buy sports-related products, or be more interested insports news, videos, or other sports-relevant, Object-linkedinformation.

Thereafter, as seen at 520, the Service can use the Objects placed atthe Markers to link Viewers together into a live social network, whereinthe Viewers are concurrently engaged in Content having the same and/orsimilar context, Objects, and/or Object metadata, for concurrent andlive communications.

One of ordinary skill in the art will recognize that numerouscombinations of subsets of the above described embodiments would also beextremely useful. For example, an airline company may want to insert anObject only for Viewers in Japan, and only during the winter, forexample to advertise discount flights from Tokyo to Australia.Therefore, the Service may recognize geographical and time-relevantattributes of Viewers selecting Content, or of Content itself, andperhaps Viewer attributes relating to income or interests, andinsert/overlay an Object into marked Content accordingly. Likewise, theService could recognize Promoter attributes, such as may relate toseasonal discounts, room availability, and/or off-peak pricing. At thesame time, however, Content Viewers in Maine may receive Objectsadvertising vacation opportunities in Arizona, as indicated by time,geographical, Viewer, Promoter and/or other relevant attributes. Otherembodiments likewise include combinations of two or more attributetypes, to specifically target information according to the specific orgeneral needs of a Promoter.

8) Marker Persistence in Distributed Content (Authorized andUnauthorized)

It is almost inherent in the nature of the internet that Content postedto even a website is frequently copied to other websites, downloaded byinternet users, modified and reposted, and generally distributed bothwith and without authorization from the Content owner. Embodiments ofthe invention take this transient nature of Content into account, andprovide benefits to Promoters, Viewers, Users, and the Service despitesuch occurrences.

For example, consider a User who posts marked Content at a Host (e.g.,YouTube, Kodak Gallery, MySpace), at 602. The User may intend for, orexpect that, Viewers will download and share the Content. For example,the Content downloaded by a first Viewer may be conveyed to a secondViewer, who may convey the Content to a third Viewer, and so on. Thus,the User has authorized, either by intent or acquiescence, that theContent be distributed to multiple Viewers (n-Viewers). In suchsituations, although the Content may be copied from the Host anddistributed from each n-Viewer to subsequent n-Viewers, the Markers andunique identification associated with the Content during Contentregistration remain associated with the Content. Thus, each time ann-Viewer selects the Content for viewing, at 610, the Service detectsthe selection at 612, identifies the Content, and inserts/overlays theappropriate Objects at each Marker in the Content.

The User may also provide authorization (e.g., express or impliedpermission) allowing n-Users to edit Marked content Therefore, anauthorized n-User may edit Content, at 608, by removing portions, bycombining other content into the Content, by copying portions of theContent into other content, or in other ways. However, when the editedContent, or other content including portions of the edited Content areselected by an n-Viewer for viewing, at 610, the Service will detect theselection. The Service can then identify the remaining Content, andinsert/overlay appropriate Objects, at 612, at the Markers in theremaining portions of the Content. The Service is able to identifyedited portions of Content because the unique Content identification, inembodiments, is included in or with a Marker. The Markers, and anassociated unique identification may be considered linked and/or‘persistent’. Wherever the Marker goes, so goes the unique Contentand/or Marker identifier. As long as an edited portion of Contentretains at least one Marker, the Content remains identifiable by theService, and Objects can be inserted/overlaid at the Marker.

Likewise, an unauthorized n-User may ‘rip’ Content, at 606. ‘Ripping’generally means that the n-User does not have authorization, eitherexplicitly or impliedly, to copy and/or edit the Content. In many cases,copying and/or editing the Content may be prohibited by contract, bylaw, or by another source of authority. In such situations, or in othersreasonably similar, the n-User may be described as an unauthorizedn-User. Subsequently, the unauthorized n-User may view the Contenthimself (as an n-Viewer), or may redistribute the Content to othern-Viewers. However, as in the case of an authorized User, when theedited Content, or other content including portions of the editedContent are selected by an n-Viewer for viewing, at 610, the Servicewill detect the selection, identify the remaining Content, andinsert/overlay appropriate Objects, at 612, at the Markers in theremaining portions of the Content.

Generally, a system is anticipated for supplementing device-accessible,viewer-perceivable content utilizing plural service providers. Such asystem can comprise one or more services configured to associate objectswith markers identified to the first service. An action of a viewer withrespect to content including a marker (e.g., downloading the content,selecting for viewing, saving, bookmarking, etc.) causes notification ofone or more such services of the action. In response to thenotification, the service(s) can associate an object with the marker,enabling the viewer to select or take some other action relative to theobject when perceiving the content.

In situations where content includes only markers identified to oneservice (e.g., registered by or to the service) is acted upon by aviewer, that marker typically receives association with one or moreobjects from only that service. However, when content includes markersidentified to more than one service, each marker in the content canreceive association with objects by whichever service the marker isidentified. Therefore, content can receive object/marker associationsfrom more than one service in response to a single action taken by aviewer relative to the content.

In still another embodiment, content may include one or more markersobtained from a third party (e.g., perhaps not a ‘service’, but rather aprovider of custom markers) and identified to one or more services. Whensuch content is acted upon by a viewer, the markers can receiveassociation with objects via any of multiple service providers to whomthe markers are identified. Thus, an embodiment of the inventionanticipates a secondary market for vendors or other providers of custommarkers. The markers can then be registered with one or more services bythe vendor or other provider of the custom markers, by a purchaser orother receiver of the custom markers, or by a User placing markers intoContent.

As described, although marked Content is distributed and re-distributedafter being transferred to a Host by an original User, and may also beedited by both authorized and un-authorized n-Users, the Markers remainpersistent in Content. Therefore, each time the marked Content orportions thereof are selected for viewing by an n-Viewer, the Service isable to detect the Content selection, and appropriately insert/overlayObjects at Markers in the Content (e.g., as relevant to time,geographical location, supply/demand curves). The n-User who ‘rips’Content may insert alternative Markers in the Content and register thosealternative ‘Markers’ with an alternative Host and alternative Servicethat will be able to insert/overlay alternative Objects in the ‘ripped’Content. The ‘ripped’ Content can allow for insertion of Objects fromthe original Host and the original Service in any of the originalMarkers that remain in the ‘ripped’ Content and also insert alternativeObjects in the alternative Markers as described herein, from thealternative Host and the alternative Service.

Associating Data Packets with Content provides Users with another methodto ensure the persistence of supplemental informational value inContent. As described, Data Packets generally carry with them adiscrete, complete set of information enabling a particular function orresult. As such, Data Packets can be excised along with a portion of theoriginal Content by an n-User, and will generally retain fullfunctionality. Therefore, if the n-User republishes a portion of Contentincluding a Data Packet associated with the original Content by theUser, the Data Packet will still function as designated by the User.

In some embodiments, however, a User may wish to set Data Packetparameters such that the packets cease to function if separated from allor a designated portion of the original Content. Such embodimentsprovide Users with some measure of control by preventing data includedin a Data Packet from being associated with Content that is notcongruent with the nature of association intended by the User. Forexample, if a User so designated, an n-User would not be able toassociate a Data Packet related to ecclesiastical material withpornographic content. Thus, the Data. Packet will not, for example,extend invitations to pornography viewers to join an ecclesiasticalsocial network, or vice versa.

9) identification of Objectionable Content and/or Objects; Denial of Useof Service

Promoters may not want their brand names, products, Objects, or identityassociated with Content which includes objectionable material.Therefore, once the User transfers Content to the Service servers, at702, during registration for using the Service to associate Markers withthe Content, the Service can review the Content for the presence ofobjectionable material, at 704. This can be accomplished by human reviewof the Content, or by using a pattern recognition system (e.g.,software). For example, representatives of the Service (or certain ofits Viewers or Promoters) may visually detect the presence ofobjectionable material. If objectionable material is identified in theContent, at 706, representatives of the Service (or certain of itsViewers or Promoters) can deny use of the Service and its tools andcapabilities (e.g., Marker and insertion processes) with regard to theContent including the objectionable material at 708. In this manner, theService can ‘filter out’ Content containing objectionable material, andprevent Service Markers and Promoters' Objects from being associatedwith objectionable material.

Alternatively, the Service may be able to provide an option to the user,a third party, and/or an automatic or selective software program toinsert clothing or some other cover over a partially or fully nude body,or other objectionable material in the Content.

Additionally, the Service can detect (by various means includingpossibly by a software program, by visual scans by representatives ofthe Service, by Viewers, or by Promoters) and deny use of the Service toContent including depictions of, or Content which in and of itselfconstitutes certain illegal activities, including, but not limited to:

-   -   a) content or activities that constitute or cause, directly or        indirectly, a breach of any contractual or other duty owed by        the User to a third party;    -   b) content or activities that constitute a tort or other legal        wrong of any kind against a third party;    -   c) content or activities that constitute a violation of any        national, state or local law, regulations, ordinance or other        applicable restriction;    -   d) content or activities that constitute or involve computer        hacking, the creation of computer ‘viruses’ of any kind or        description, and/or the negligent, reckless or intentional        distribution of such viruses; and/or    -   e) activities that involve, foster, or are in the nature of bulk        emailing, ‘spamming’, or similar mass distribution activities.

The Service can detect (by various means including possibly by asoftware program, by visual scans by representatives of the Service, byViewers, or by Promoters) and deny use of the Service to Contentincluding depictions of, or Content which in and of itself constitutescertain potentially harmful activities, including, but not limited to:

-   -   a) tobacco, alcohol, and or drug use;    -   b) use of weapons;    -   c) gambling;    -   d) content that sets a bad example for young children, and/or        that teaches or encourages children to perform harmful acts or        imitate dangerous behavior;    -   e) content that creates feelings of fear, intimidation, horror,        and/or psychological terror; and/or,    -   f) incitement or depiction of discrimination or harm against any        individual or group based on gender, sexual orientation, ethnic,        religious and/or national identity.

Additionally, the Service may identify (by various means includingpossibly by a software program, by visual scans by representatives ofthe Service, or by Viewers, or by Promoters) Content that is libelous,defamatory, threatening, abusive, fraudulent, harassing, that violatesthe privacy rights of any individual, that is of a political nature, orthat is copyrighted, trademarked, patented or otherwise protectedcontent for which there is no prior express written consent for use fromits owner.

In much the same way as described above relative to Content, Users,Hosts, Viewers, and a Service itself may not want objectionable materialinserted and/or overlaid at Markers in Content. This is particularlytrue considering that Content may be of a nature that young children maygenerally view, for example, and the objectionable nature of materialinserted/overlaid within the Content may not be apparent until a childhas already been exposed to it. Therefore, a Service may also wish toreview Objects for objectionable material before allowing them to beinserted and/or overlaid at Markers. This may generally be unnecessarydue to contractual provisions between a Service and a Promoterprohibiting such material in Objects or linked to by Objects. However,such capability provides an added layer of protection in case of breachof such a contract, or some other situation in which objectionablematerial may otherwise be inserted and/or overlaid in marked Content.

10) Control Over Content, Markers, and Inserted/Overlaid Objects:Flagging and Digital Rights Management Controls

The Service can exert control over Content, Markers, andinserted/overlaid Objects in several ways. One method of controlincludes providing tools and capabilities to Viewers to ‘flag’ problemswith Content or inserted/overlaid Objects. With reference to FIG. 8, aViewer may identify objectionable material in marked Content, at 802, orinappropriate marking of Content, at 804, such as Markers that are notput in the right place or in the right context within Content.

In either case, the Service can provide the Viewer with capabilities toinitiate ‘flagging’ of the suspect Content and/or Marking, as at 806.Flagging may be initiated, in embodiments, by selecting an icon in atoolbar, by opening an email or other messaging dialogue to the Service,or by some other method or tool. However activated, the Viewer-initiatedflagging causes the Service to become aware of a Viewer-perceivedproblem with marked Content. Therefore, at 808, the Service detects theViewer-initiated flagging, and either immediately or at some later timereview the Content and/or its associated marking for problems. Theflagging may allow the Viewer to provide information identifying and/ordescribing the general or specific nature of the perceived problem. Forexample, the Service may provide a list of selectable categories ofgeneral problem types, or may provide a text entry box into which aViewer can describe the problem from the Viewer's perspective.Additionally, the flagging process may ‘capture’ the unique Contentidentification of the flagged Content so that the Service can readilyand accurately associate the reported problem with the correct Content.

As mentioned, in general, the Service may not retain Content on Serviceservers after the Content is marked by a User. Therefore, the Servicecan either review the Content by accessing it where it resides at theHost, or cause the Content to be transmitted or copied back to theService servers to be reviewed there.

After reviewing the Content in light of the Viewer-initiated flagging,if the Service determines that a corrective action is warranted at 810,the service can then implement an appropriate corrective action at 812.Examples of corrective actions can include, but are not limited to,removing or deactivating all or a subset of the Markers in the Content,deleting inserted/overlaid Objects from Content, fixing or prompting aUser to fix the placement of all or a subset of the Markers in theContent, deleting or requesting a Host and/or User to delete Content orto otherwise prevent Viewer access to Content, and/or or other actionsas deemed appropriate and/or effective to mitigate the problem at hand.In a situation where multiple Objects are inserted/overlaid at a Marker,and only a subset of the Objects is/are related to objectionablematerial, any one or more of the Objects can be removed from the Markerwhile leaving the other Objects in place. Thereafter, the Service canterminate the Viewer-initiated flagging, at 816.

Alternatively, after reviewing the Content and/or associated flagging inlight of the Viewer-initiated flagging, the Service may determine thatno corrective action is required, as shown at 814, and thereforeterminate the Viewer-initiated flagging, at 816. Whether or notcorrective action is taken, the Service can notify the Viewer regardingthe determination and result.

Another type of control that Service can employ for Content, Markers,and insertions/overlays is Digital Rights Management (DRM). DRM can beused to protect marked Content, active and/or interactive, so that itcannot be altered later. For example, DRM can prevent marked Contentfrom being later edited to include or be included in objectionablematerial. With reference to FIG. 9, after a User associates Markers withContent, at 902, the Service can apply DRM to the marked Content, at904. Thereafter, if an n-User, or even the User himself, breaches theDRM of the marked Content, at 906, the Markers associated with theContent become inactive, at 908. This may mean that the Markers can nolonger receive inserted and/or overlaid Objects. It may also mean thatany Objects already inserted/overlaid at Markers in the Content aredeleted from the Content.

11) Promoter Control and Updating of Inserted and/or Overlaid Objects

Embodiments of the invention provide Promoters with capabilities tocontrol which Objects are inserted/overlaid at Markers, and allowupdating of inserted/overlaid Objects to meet the Promoter's needs. Withreference to FIG. 10 at 1002, after a Promoter has provided the Servicewith access to a first Object for insertion/overlay in marked Content, aPromoter may identify or select a second Object to replace the firstObject. Therefore, the Promoter disables capability forinsertion/overlay of the first Object, at 1004, and enables capabilityfor insertion/overlay of the second Object, at 1006. Disablinginsertion/overlay capability may involve removing the first Object fromthe Promoter's server, a Service server, or a third party server, orremoving the Service's access to the first Object on the Promoter's or athird party's server. Likewise, enabling insertion/overlay capabilitymay involve adding the second Object to the Promoter's server, a Serviceserver, or a third party server, or providing the Service with access tothe first Object on the Promoter's or a third party's server. The secondObject may be provided to the Service or a third party by other means aswell, such as by providing storage media including the second Object,although other alternative may also be used.

Once capability for inserting/overlaying the second Object is enabled,when a Viewer selects marked Content for viewing at 1008, the secondObject rather than the first Object is inserted and/or overlaid at aMarker in the Content.

The ability to update Objects can likewise include Objects alreadyselected by Viewers and stored for later review. Further detail on thiscapability is provided in FIG. 11 and described below. As shown at 1102,a Viewer selects a first Object in marked Content, and the selection isstored so that the Viewer can access and/or review the Object at a latertime. Alternatively, the Viewer may have downloaded and stored theContent itself without selecting a specific Object, although the Contentalready includes Objects inserted at Markers.

Sometime after the Viewer selects a first Object or downloads theContent, the Promoter decides disables capability for the first Objectto be inserted/overlaid, at 1104, and instead, enables capability forinsertion/overlay of a second Object. The Service, upon some pre-settime interval or some triggering action (e.g., Viewer connects toInternet, Viewer selects stored Content for viewing, Promoter sendsrequest), alters the Viewer's stored selection so that the first Objectis replaced with a second Object, at 1108. When the Viewer then accessesand/or reviews the stored selection (or Content), at 1110, the Viewer isprovided with the second Object and its associated data rather than thefirst Object. The System may detect that the Viewer has limitedbandwidth for transferring information, and may regulate the rate atwhich Objects are downloaded to the Viewer to prevent overloading theViewer's bandwidth.

With the capability to update Objects as described, Promoters can ensurethat Viewers receive the latest, most up-to-date and most relevantinformation that a Promoter wishes to provide. Further, Viewers canobtain updated information from previous selections, eliminating theneed to access Content and repeat selections each time the Viewer wishesto learn about, for example, a Promoter's latest sale items, updatednews stories, or promotional discounts or packages.

12) Insertion of Virtual Environments for Advertising, AdditionalContent Viewing and Sharing, and Social Networking

Virtual environments represent another class of embodiments of theinvention that a Service can enable for Users, Viewers, and Promoters,to further enrich a Viewer's Content-based experience.

With reference to FIG. 12, once a User registers Content with a Service,at 1202, the Service provides the User with the capability to associatevisible depictions of code (‘Code’ hereinafter) (e.g., icons) with theContent, at 1204. As with Marker data, a Service will receive from aUser and retain a record of data relating to the placement of Code,including characteristics relating to the placement of the Code relativeto the Content. The Code enables a Viewer to see Objectsinserted/overlaid in such virtual environments as virtual product rooms,virtual meeting rooms, virtual social networks (including as detailed inthe above referenced provisional patent application), or other virtualenvironments, whether live or static. Such virtual environments caninclude capability for voice over internet protocol (VoIP)communications, text messaging, or other means of communications betweena Viewer and others.

By associating the provided Code with Content, at 1206, the User enablesvirtual environments to be inserted/overlaid at a Marker associated withthe Content, or elsewhere such as at the beginning or the end of theContent (e.g., a video), in a Content title, or somewhere within theContent (as described in U.S. Pat. No. 7,168,051 referenced above). Avirtual environment, once activated by a viewer by selecting an icon orother Code at 1208, can exist as a virtual world overlay on top of thevisible Content, or as an environment that is separate from the confinesof the visual Content.

Viewer-selected depictions of code can, in an embodiment, be used tolink the Viewer into a live social network with other Viewers who aresimultaneously engaged in Content having the same/similar context ormetadata of the selected Code, or who are engaged in the same/similardepiction of Code, for live (e.g., concurrent, contemporaneous,simultaneous) communications with each other.

For example, if the Content is a digital photo, selecting (e.g., mouseclicking on) the Code icon, or perhaps clicking on the photo imageitself, will expose to the Viewer a virtual environment around, on, orwithin the photo. If within the photo, the virtual environment couldhave spatial dimensions such that an image of the photo is visible tothe Viewer, for instance on a wall of a virtual room. Thus, at 1210, theViewer perceives the virtual environment associated with the Content.Content displayed within a virtual environment can includeadvertisements and/or other branded items having some relevance toand/or representation(s) of a promoter's products, commercial identity,or message, and will generally be intended and/or configured to attracta Viewer's interest.

A virtual environment, according to embodiments of the invention, allowssimultaneous access to more than one Viewer. A Viewer will be able tosee and interact with other Viewers of the Content who, by alsoselecting associated Code, can also enter into the Virtual environmentwith the Viewer. Each Viewer in a common virtual environment can berepresented visually by either a live or static image, avatar, or someother graphic or pictorial identification that others in the virtualenvironment can see. Alternatively, Viewers in a common virtualenvironment can be invisible to each other, and/or can permit or preventother Viewers from interacting with them.

Once a Viewer activates a virtual environment, as at 1302 in FIG. 2,numerous options may be available. At 1304, the Viewer can view activeand/or static content in the virtual environment, and at 1306, selectObjects inserted/overlaid within the virtual environment. For example, aViewer viewing Content may select an Object which takes the Viewer to aMcDonald's branded social networking room. The room could containMcDonald's branding throughout, including further selectable Objectslinked to McDonalds related brand and/or product information. In anotherexample, a Viewer could select an Object branded with a clothing brand(e.g., Versace, Gap), which would open a virtual environment. The Viewercould then take a virtual walk through a store, and even try on clothingusing a virtual mannequin having/representing the Viewer's physicaldimensions (e.g., height, weight) and/or the Viewer's face/head. TheViewer could interact with other Viewers also in the virtualenvironment, as at 1308, getting recommendations for links to otherclothing websites, talking with company representatives and/or salespeople, meeting new friends, and carrying on other social networkingactivities. Additionally, selecting or otherwise interacting with abranding Object in a virtual environment can link the Viewer with others(e.g., whether also inside the virtual environment, outside it, or both)based on matching and/or similar metadata descriptors associated withthe Object.

If, for example, a Viewer selects and enters a virtual environment for adepartment store, the Viewer can select Objects inserted/overlaid withinthat virtual environment which take the viewer into other virtualenvironments. For example, the Viewer may want to wander back and forthbetween the department store virtual environment and virtualenvironments of various merchandise companies, by selecting Objectsrepresenting the merchandiser's brand or products. The Viewer can eveninvite other Viewers in the first virtual environment to come with theminto the other virtual environments, as at 1314. In this instance, theinvited Viewer, accepting the invitation, can cede control, allowing theinviting Viewer to take both viewers into the second virtual environmentsimultaneously.

A virtual environment can include a plurality of virtual rooms (whichcan also be scenes, environments or other depictions of an n-dimensionalenvironment wherein ‘n’ is greater than or equal to 2, and a Viewer can‘move’ about within the virtual environment, at 1312, such as from oneroom to another. This may involve selecting Objects representing ‘doors’to other rooms, or a Viewer may be able to select and drag or otherwisemove a representation of themselves (e.g., an avatar) in a virtualenvironment through a virtual environment representation of a doorway,or some other method. The movement of the Viewer may be visible to otherViewers in the virtual environment. Likewise, the Viewer may be able tosee other Viewers, or viewable representations thereof, moving aboutwithin the virtual environment, much as if all the Viewers were in areal world environment. Users can move within the virtual environmentand move to and within other virtual environments in an expansivevirtual world.

Each virtual room can contain movie screens, areas for displayingphotos, videos, and/or text content, and/or inserted/overlaid Objectsthat link to Other types of Content. The Viewer could select and play avideo on one screen, while viewing photos on another area of the virtualroom, and simultaneously selecting playing uploaded music on a virtualaudio sound system embedded into the virtual environment. The Viewercould invite other Viewers in the virtual room to enjoy the sights andsounds, and can converse with them via email, text messaging, voiceand/or video chat, VoIP, or any of a number of other communicationsmethods. A Viewer, while in a virtual environment, can also contactfriends or associates who are not simultaneously in the same virtualenvironment, and invite them in to share a rich virtual environmentexperience, as at 1310. For example, in addition to the communicationsmethods listed above, the Viewer might select and drag a listing for afriend out of a ‘buddy list’ and into the virtual environment. As aresult, a message may be sent to their friend inviting them into andproviding information on how to access the virtual environment.

The effect of a virtual environment, as a shared social andentertainment experience, is that a Promoter can create a relatively‘captive’ environment, immersing Viewers in a fully branded environmentreplete with the Promoter's identity and products. The longer a Viewerstays inside a virtual environment, the longer he is exposed to brandnames or branded objects, and the more completely and effectively aPromoter can convey information to the Viewer. Therefore, in anembodiment, a virtual environment can present a collection of commercialtransaction access points and/or commercial exposures, such as a virtualtrade show, virtual mall, virtual department store, virtualentertainment complex, virtual commercial directory, or any combinationthereof. In one example, in a real estate context, a real estate agentcould take a client on a virtual tour around the exterior and/orinterior of multiple homes, and can refine presented virtual environmentbased upon the client's desired features (e.g., number of bedrooms,etc.). Selecting different doors in a virtual environment can take theclient to different properties, to different sales agents, to differentgeographical markets, etc.

Similarly to how Viewers can be invited into a virtual social networkingenvironment, as already described, Data Packets associated with Contentcan serve the same or a similar purpose. Thus, a Viewer selecting a DataPacket may be invited into, or perhaps taken directly to, ann-dimensional (e.g. 2, 2-1/2D, 3D, etc.) environment including avatarsof other already, present members/participants. Upon entering, there maybe a welcome message and instructions on how to participate inside thatenvironment. These instructions may include information on how to assumean avatar form, how to chat with other participants, how to move aboutwithin the environment, and other activities. Likewise, entry of theViewer into the environment could activate a signal to otherparticipants who may want to initiate interaction with the Viewer.

As shown at 1316, once the view activates a Virtual Environment, theViewer can be concurrently linked with other Viewers in a live socialnetwork, wherein the other Viewers are concurrently engaged in the sameand/or similar Virtual Environment, context, or Content.

13) Generating and Distributing Revenue Gained Through Insertion/Overlayof Content

A system as described in an embodiment herein therefore also includes arevenue collection mechanism. Generally speaking, the revenue collectionmechanism identities instances of specified payment-obligation-incurringactions which, when taken by certain entities, for the basis for apayment obligation. Several examples of ‘specified actions’ aredescribed below. A revenue collection mechanism, upon identifying anincidence of a specified action, creates a record indicting one or moreof a payment obligation, an entity obligated to provide payment, and oneor more entities entitled to receive all or a portion of the payment.Such record can be stored at a memory (data storage) means of a centralserver or otherwise.

There are numerous ways by which a Service can generate revenue throughthe numerous embodiments described herein. The Service can chargePromoters to place their Objects at the Service's Markers. Charges canvary based on the number of Objects inserted/overlaid, or the durationof time over which a Promoter has access to Markers for insertingObjects. The Service can receive payments from a Promoter based on theamount of sales and/or profit the Promoter generates through Viewersselecting inserted/overlaid Objects, or the total number of ‘hits’ aPromoter's Objects receive, regardless whether sales result from those‘hits’. Promoters could be charged for access to different geographicareas. Charges could also increase for placing Objects at Markers inhighly popular Content, or Content accessible through particular Hosts.These are only a few examples, and should not be taken as limiting thepossible revenue generating embodiments in any sense.

With regard to distribution of revenues or payments of other types,there are also many options conceived with regard to embodiments of theinvention. For example, Viewers could be paid to recommend Objects toother Viewers, for example by using a tool and/or capability provided bythe Service. A Viewer, before, during, or after selecting an Object, orwhile viewing Content, could select an icon or other command, and send amessage to another Viewer recommending the Content and/or Object. Themessage would have information associated and/or embedded within itwhich would inform and/or provide the receiving Viewer with informationguiding them to the recommended Content and/or Object. The payment maybe based solely on the recommendation, and/or may include some paymentfor any sales that result from the recommendation. Viewers may alsoreceive payment or some other value for identifying and flaggingproblems with Content and/or Objects inserted/overlaid in Content.

Users, Promoters, or others could receive payment for any revenue and/orprofit resulting from Objects inserted/overlaid at Markers the Userassociated with Content. This serves as an incentive for Users to markContent, and could form the basis of a new business model. Additionally,Users, Promoters or others could receive revenue based on the amount oftime Viewers spend in virtual environments/worlds (VE) triggered byselection of an inserted/overlaid Marker, Object, Data packet, Icon, orother similar code, or selection of inserted/overlaid Markers, Objects,Data packets, Icons, or similar code within a VE initiated by a Vieweraction relative to marked Content.

Hosts may be compensated for referring Users to the Service, and forproviding the Service with functional access to marked Content locatedon their server(s) and/or websites. Functional access may include, asdescribed above, the ability to edit Content or Markers in Content, forexample when offensive material is detected in Content through flagging.Likewise; the Service may need to be able to remove the Content entirelyfrom the Host site and/or return it to affect corrective actions relatedto Content.

Objects in content or in a virtual environment could also be used toprovide a Viewer with coupons for products or services. For example, theViewer could select and ‘open’ a virtual refrigerator or cabinet and bepresented with one or more Objects (e.g., food items, household items,entertainment events, etc.). The Objects may include an indication thata special offer (e.g., coupon, rebate, etc.) related to an aspect (e.g.,appearance, etc.) of the Object is available to the Viewer. For example,the Viewer may see an object appealing as a bottle of ketchup, withvisual highlighting distinguishing it from other similarly placedObjects. By selecting the ketchup bottle object, the viewer can obtain acoupon usable for a commercial transaction (e.g., a purchase, rental,lease, etc,) related to the item. The commercial transaction utilizingthe coupon could be a virtual transaction (e.g., in a virtual world suchas Sim City), an on-line actual transaction (e.g., at Amazon.com), areal-world physical transaction (e.g., at a grocery store), or any othercommercial transaction. Revenue could be generated by obtaining paymentfrom Promoters for, placement of objects providing access to coupons,etc., such as by auctioning object placement to the highest bidder.

In general, embodiments of the invention provide the capability to, andindeed anticipate, a broad range of new business and revenue producingmodels not limited to those examples specifically listed herein.

14) System for Creating, Distributing, Hosting, and/or Viewing MarkedContent

With reference to FIG. 15, a system is described herein by whichviewable content can be marked, distributed, hosted, and/or viewed, andobjects can be inserted and/or overlaid relative to marked content.Embodiments of a system need not include all features or elementsdepicted according to FIG. 15, and can include other features orelements as would be understood to those having ordinary skill in theart.

An embodiment of a system can include between one and n-servers, where‘n’ can represent either a fixed or variable number of servers in asystem. A Service server 1502 can retain Marker-relevant data,Content-relevant data, and User-relevant data, as well as other datauseful to facilitate placement and use of Content Markers according toembodiments of the invention. A User server 1506 may be present forposting Content for distribution, such as when a User is also a ContentHost, although it is not limited to only such situations. A Host server1508 may be present in embodiments to provide hosting for markedContent. An example of a host server would be a server maintained by awebsite provider (e.g., YouTube, Google Images, Kodak Photo Gallery, agovernment agency, an educational resource, or others).

An embodiment of a system may include a Promoter server 1510 whereObjects can be retained, or from which Objects may be provided to theService for inserting and/or overlaying relative to marked Content. Athird party server 1512 may be present, and may provide data retention,transfer, processing or other functions or services on behalf of a Host,Promoter, Service, User, or Viewer.

Any and/or all servers in a system according to embodiments willgenerally, but not necessarily, be connected in some form to theinternet 1500 (or another network). Connection to the internet 1500 maybe provided through a third party internet service provider (ISP), orthrough a proprietary internet service provided and/or controlled atleast in part by a Host, Promoter, Service, User, and/or Viewer.Connections may be wired or wireless, and may utilize any of thenumerous wireless technologies (e.g., 802.11a, 802.11g) and/or wiredtechnologies (e.g., high speed electrically conductive cable, telephonemodem, fiber optic cable) currently available and/or known to thosehaving skill in the art. Various equipment and/or devices such as.routers, repeaters, modems, relays, switch boxes, transmitters, networkcard devices, and other devices may also be utilized to facilitateand/or enable data transmission between servers (and/or devices) and theinternet, and with other servers (and/or devices) through the internet.Any such equipment and/or devices, either individually or collectively,can be considered a data transmitting means in an embodiment.

In addition to servers, a system according to various embodiments alsoincludes devices for creating, marking, storing, viewing, editing,and/or otherwise transmitting, receiving, and/or utilizing markedContent, although the use of devices is not so limited. A User device1504 and/or an n-User device 1518 provides numerous functions asdescribed above with regard to creating and/or marking Content, anduploading Content, marked or unmarked, to a Service and/or a Host. AUser device 1504 and/or an n-User device 1518 can also be used forviewing marked or unmarked Content, editing Content, selecting Objects,and processing Object-linked data, among numerous other uses andactivities.

A Viewer device 1514 and/or an n-User device 1516 can likewise be usedfor viewing marked or unmarked Content, editing Content, selectingObjects, storing selections, and processing Object-linked data, amongother uses and activities. A Viewer device 1514 and/or an n-User device1516 can also provide for communicating with other Viewers and/orn-Viewers, via email, text messaging, instant messaging, voicemessaging, VoIP, or other communications technologies and/or methodscurrently in use or known to those having ordinary skill in the art.

Viewer devices 1514 and/or User devices 1516 may transfer data withand/or via the interne by any of the devices, technologies and/ormethods listed above regarding servers, as well as by any othertechnologies and/or methods currently in use or known to those havingordinary skill in the art. However, viewer devices 1514 and/or Userdevices 1516 may also operate temporarily or for extended periods oftime without having an active connection with the internet, either wiredor wireless. Thereafter, such devices can then be connected with theinternet, and data can be transferred with and/or via the internet.

A User and/or a Viewer device will typically include some type ofsoftware generated ‘desktop’ comprising all or some portion of theviewable display of the device. The desktop may be functionally enabledby software resident on the device (e.g., Microsoft Windows desktop, MacOSX desktop, or others), or on a peripheral connected device.Alternatively, it may be a web desktop (a.k.a. ‘webtop’ or ‘virtualdesktop’) enabled at least in part by a remotely located software sourceacting over the internet and interacting with browser software (e.g.,Microsoft Internet Explorer, Safari, FireFox, Netscape Navigator, orothers) on a device.

It should be clear, therefore, that any device and/or server listed orimplied to exist in the embodiments described herein may be connectedwith any other device and/or server listed or implied to exist hereinvia permanent, lasting, temporary, and/or occasional connection withand/or through the internet. Further, Service-related activities can beenabled and can transact via an entirely web-based interaction. Forexample, all software, files, programs, and/or capabilities of a Serviceor otherwise useful to practice the invention can be accessed andemployed by a User, Promoter, Viewer, n-User, or n-Viewer via theinternet or another network. This arrangement enables practice of anembodiment of the invented methods from any internet-linked or linkabledevice, no matter where located. For example, a User need not use thesame device or access a Service from the same location at each instance,but can remain mobile. Also, an embodiment of the invention can bepracticed on, by, or through any internet-linked or linkable device,even if the device has not been prepared in any particular way for suchpractice. For all practical purposes, even a device which has nocapability for storing files, software instructions, Content, etc., canpractice an embodiment of the invention. This is because any and/or allinstructions, files, Content, etc. can be accessed and executed via theinternet, and a beneficial result can then be transmitted to a Viewer orother for Viewing and/or other purpose at the device.

User and/or n-User devices (1504 and 1518, respectively, and Viewer andn-Viewer devices (1514 and 1516, respectively) may be any electronicdevice capable of and/or aiding in viewing, processing, storing,editing, manipulating (e.g., altering frame rate, image contract, tonequality), transferring, or otherwise using marked and/or unmarkedContent, Objects, and/or Object-linked information. Examples include,but are not limit to, personal computers (e.g., desktop, mobile,palmtop), personal data assistants (PDA), electronic game systems (e.g.,Playstation III, X-box 360, Nintendo Wii), mobile telephones,televisions, digital video recorders (DVR), satellite enabled devices(e.g., navigation, radio, communication) and other devices orinstruments. This includes such devices and/or instruments whether theyare generally considered consumer, commercial, enterprise, scientific orindustrial in nature.

Devices and/or servers according to embodiments of a system describedherein also generally include peripheral input and/or output devices.Examples of such I/O devices may include a keyboard, mouse, display(‘monitor’), wireless card, antenna, printer, facsimile, scanner,enabling device (such as a may be visually, physically, and/or hearingimpaired), port replicator, docking station (e.g., for media playersincluding iPod, for mobile phones, for mobile personal computer),although the embodiments are not so limited.

Likewise, devices and/or systems described herein may be capable ofoperation utilizing various operating systems, including but not limitedto those provided by or based upon Microsoft, Apple, Novell, Linux,Unix, or other commercial, open, and/or proprietary technologies, andwhether 32-bit, 64-bit, or some other architecture currently availableor know to those having ordinary skill in the art.

As a relatively centralized data-handling entity, a Service can send,receive, accommodate, interpret, store, compile, connect, recognize,manipulate, alter, process, pass-through and/or ignore numerous typesand quantities of information to and/or from numerous types andquantities of information sources and/or targets (e.g. requestors ordesignated recipients). Other data-handling activities are alsopossible. A Service can recognize and record transactions, and provide arecord for later retrieval or other action. A Service can recognizeMarker, Object, and data packet related activity whenever a device orentity undertakes such activity (e.g., selection, execution, placement,association, or others) and the device or entity is concurrently, orsubsequently becomes, electronically connected with the internet.Likewise, a Service can retain a record of every Marker, Object, and/orData Packet associated with Content, as well as the information (e.g.,textual, graphic, audio, or other) comprising an Object or Data Packet,and various other relevant information (e.g. temporal information, Userinformation, or other information) related to the same.

Therefore as shown in FIG. 16, Viewers in live social networks and/orn-dimensional virtual environments can communicate with each other,share advice, trade content files and partake in other interactiveactivities involving ‘information’, as at 1602, and a service can recordand store some or all of the information, as at 1604. At 1606, Viewerscan access this stored (historical) information at the same time or at alater time when selecting Objects, Markers, or Content, such as whenentering into a specific social network and/or a specific virtualenvironment within which and/or from which such information originated.As shown at 1608 and 1610, respectively, the accessed information can beuse by Viewers to create live social networks and/or Virtualenvironments, and a Service can use the information for commercialpurposes. Therefore, storage and availability of information arisingwithin social networks and/or n-dimensional virtual environments forconcurrent and/or later use is contemplated within an embodiment of theinvention.

Therefore, as described herein, the embodiments of a system forcreating, distributing, hosting, and/or viewing marked Content, amongother activities and/or operations described herein, may be quite broadand inclusive.

If will be understood that the present invention is not limited to themethod or detail of construction, fabrication, material, application oruse described and illustrated herein. Indeed, any suitable variation offabrication, use, or application is contemplated as an alternativeembodiment, and thus is within the spirit and scope, of the invention.This can include altering the sequence of operations depicted in thefigures according to various embodiments, as well as the sequence ofvarious figures should not be interpreted as limiting the scope,character, operability or utility of alternative embodiments of theinvention.

From the foregoing, those of skill in the art will appreciate thatseveral advantages of the present invention include the following:

Embodiments of the present invention provide Content creators (Users),Hosts, and Promoters with new and highly adaptive opportunities togenerate revenue streams. Embodiments of the invention have therealistic potential to enable entirely new and innovative businessmodels.

Embodiments of the present invention prevents Content from quicklybecoming irrelevant and uninteresting, and allows Promoters to targetinformation to those Viewers most likely to have an interest in theirinformation.

Embodiments of the present invention provide the ability to preventPromoters' names, brands, products, and information from beingassociated with objectionable material, thereby providing the ability toprotect their image while simultaneously increasing their marketingreach and effectiveness.

Embodiments of the present invention provide Content Viewers with readyaccess to highly relevant, interesting, fresh, and helpful information,and keep Content interesting. In essence, Content becomes a dynamicinformation source and a rich experience, including even an access pointto interactive virtual environments populated by others sharing similarinterests.

Embodiments of the present invention cumulatively possess the potentialto save all parties involved, Promoters, Users, Viewers, Hosts, andothers enormous amounts of at least two precious resources; time andmoney. This is accomplished by enhancing of relevance of Content, whichcan be tailored by Promoters, Users, Viewers, and perhaps even Hosts, toconnect those who want with those who have, quickly and efficiently.

Embodiments of the present invention also provide a system and a methodfor enabling the above mentioned advantages, including a Service thatprovides the enabling tools and services. Thus, the Service provides theability to reshape the way people interact with each other, withproviders of mod, services, and information, and with the Content thatso strongly influences social culture.

The advantages listed here do not constitute an exclusive list, norshould they be interpreted, either individually or collectively, as astatement of the complete intents and/or purposes of the presentinvention.

It is further intended that any other embodiments of the presentinvention that result from any changes in application or method of useor operation, method of manufacture, shape, size, or material which arenot specified within the detailed written description or illustrationscontained herein yet are considered apparent or obvious to one skilledin the art are within the scope of the present invention.

Finally, those of skill in the art will appreciate that the inventedmethod, system and apparatus described and illustrated herein may beimplemented in software, firmware or hardware, or any suitablecombination thereof. Preferably, the method system and apparatus areimplemented in a combination of the three, for purposes of low cost andflexibility. Thus, those of skill in the art will appreciate that themethod, system and apparatus of the invention may be implemented by acomputer or microprocessor process in which instructions are executed,the instructions being stored for execution on a computer-readablemedium and being executed by any suitable instruction processor.

Accordingly, while the present invention has been shown and describedwith reference to the foregoing embodiments of the invented apparatus,it will be apparent to those skilled in the art that other changes inform and detail may be made therein without departing from the spiritand scope of the invention as defined in the appended claims.

We claim:
 1. A method of associating advertising materials and otherviewer-targeted data with viewable content, comprising: receiving userregistration information from a user; receiving from the userinformation identifying an item of content, wherein the content isimagery viewable at a display device coupled with a network-linkeddevice; providing to the user a marker comprising device-executableinstructions; providing to the user a device-enabled capability toassociate the marker with a user-selected position relative to aviewable portion of the content, wherein the associating includes one ormore of: dragging and dropping the marker into a specified positionrelative to a selected portion of the content via device-enabledcontrols; copying and pasting the marker into position relative to aselected portion of the content via device-enabled controls; specifyinga position within the content by data entry via an input/output device;indicating a position within the content by visibly placing amouse-controlled cursor or other similar visible indicator at theposition and activating a corresponding control to select that position;and defining a visible, user-defined outline at a portion of the contentusing device-enabled controls; and receiving from the user dataindicating a user-selected marker position relative to the viewableportion of the content.
 2. The method of claim 1, further comprising:receiving from the user registration information for content marked withthe marker; and associating a unique identification with the markedcontent.
 3. The method of claim 1, further comprising: assigning one ormore objects to be associated with the marker when a viewer views thecontent.
 4. The method of claim 2, further comprising: providing to theuser a user-selectable option to receive payment corresponding to vieweractions relative to the marked content; and receiving from the user dataindicating a selection by the user relative to the provided option. 5.The method of claim 2, further comprising: providing to the user anoption to specify one or more types of objects to be associated with themarker in the marked content.
 6. The method of claim 2, furthercomprising: providing to the user an option to positionally stabilizethe marker relative to a viewable, user-selected pattern within thecontent imagery, wherein the positional stabilization is enabled bypattern-recognition software.
 7. The method of claim 2, wherein themarker is configured to be visible to and selectable by a viewer whenthe viewer views the content.
 8. The method of claim 3, wherein themarker is configured, when selected by a viewer of content marked withthe marker, to execute coded instruction corresponding to the object ina viewer-perceivable manner including one or more of aurally, visuallyand tactilely.
 9. The method of claim 1, wherein associating the markerwith a viewable portion of the content comprises forming an additionallayer of content overlying, corresponding to, and dynamically responsiveto the imagery of the content.
 10. The method of claim 3, furthercomprising: changing an object association with the marker from oneobject to another object based on the presence of one or both of aviewer-dependent condition and a viewer-independent condition, wherein:a viewer-dependent condition is one or more of a viewer location, aviewer gender, a viewer age, a viewer financial status, and aviewer-expressed interest; and a viewer-independent condition is one ormore of an expiration of a time period, an arrival of a specified timeor date, a payment by a sponsor, an instruction received from the user.11. The method of claim 3, wherein the object is an advertisement foreither or both of a commercial product and a commercial service.
 12. Themethod of claim 3, further comprising: inserting the object at themarker in response to content marked with the marker being selected forviewing by a viewer.
 13. The method of claim 2, further comprising:analyzing metadata of a marker in response to the marked content beingselected for viewing by a viewer; identifying another viewer who iseither concurrently viewing marked content having similar markermetadata or who has left an instruction to later be connected with aviewer of marked content having similar marker metadata.
 14. The methodof claim 13, further comprising: linking the viewer and the other viewerinto a live, device-enabled social networking interaction via a datanetwork connection with each of the viewer and the other viewer.
 15. Themethod of claim 13, wherein the linking includes presenting a virtualenvironment at a display device of either or both of the viewer and theother viewer.
 16. The method of claim 15, further comprising: providingat least one device-enabled method for the viewer and the other viewerto communicate with each other within the virtual environment.
 17. Amethod of associating advertising materials and other viewer-targeteddata with viewable content, comprising: receiving user registrationinformation from a user; receiving from the user information identifyingan item of content, wherein the content is imagery viewable at a displaydevice coupled with a network-linked device; providing to the user adata packet comprising device-executable instructions; providing to theuser a device-enabled capability to associate the data packet with auser-selected position relative to a viewable portion of the content,wherein the associating includes one or more of: dragging and droppingthe data packet into a specified position relative to a selected portionof the content via device-enabled controls; copying and pasting the datapacket into position relative to a selected portion of the content viadevice-enabled controls; specifying a position within the content bydata entry via an input/output device; indicating a position within thecontent by visibly placing a mouse-controlled cursor or other similarvisible indicator at the position and activating a corresponding controlto select that position; and defining a visible, user-defined outline ata portion of the content using device-enabled controls; and receivingfrom the user data indicating a user-selected data packet positionrelative to the viewable portion of the content.
 18. The method of claim17, wherein the data packet is configured to be visibly depicted to andselectable by a viewer when the viewer views the content.
 19. The methodof claim 17, further comprising: providing to the user a user-selectableoption to specify whether the data packet will execute spontaneouslywhen a viewer views the content, or whether the data packet will executeonly in response to selection of the data packet by the viewer, andreceiving from the user data indicating a selection by the user relativeto the provided option.
 20. The method of claim 17, wherein the datapacket is configured to cause a message to be sent to the user uponeither or both of spontaneous execution of the data packet and selectionof the data packet by the viewer.